Augmented audio for communications

ABSTRACT

A device includes one or more processors configured to determine, based on data descriptive of two or more audio environments, a geometry of a mutual audio environment. The one or more processors are also configured to process audio data, based on the geometry of the mutual audio environment, for output at an audio device disposed in a first audio environment of the two or more audio environments.

I. FIELD

The present disclosure is generally related to augmented audio forcommunications.

II. DESCRIPTION OF RELATED ART

As technology advances, more and more computing environments incorporateelements of extended reality (“XR”), such as virtual reality, augmentedreality, mixed reality, etc. In XR environments, one or more virtual, orcomputer-generated, elements can be present in a user's computingenvironment. The user's computing environment can also include one ormore elements from the user's actual reality.

Advances in technology have resulted in smaller and more powerfulcomputing devices. For example, there currently exist a variety ofportable personal computing devices, including wireless telephones suchas mobile and smart phones, tablets and laptop computers that are small,lightweight, and easily carried by users. These devices can communicatevoice and data packets over wireless networks. Further, many suchdevices incorporate additional functionality such as a digital stillcamera, a digital video camera, a digital recorder, and an audio fileplayer. Also, such devices can process executable instructions,including software applications, such as a web browser application, thatcan be used to access the Internet. As such, these devices can includesignificant computing capabilities, including, for example, multimediasystems that enable the user of the device to interact with an XRenvironment.

It can be challenging to make a user's experience of the XR environmentsrealistic and seamlessly merged with the user's real-world physicalenvironment. For example, differences in acoustical characteristicsbetween two environments can cause sound output to the user to seemunnatural.

III. SUMMARY

In a particular aspect, a device includes a memory storing instructionsand one or more processors coupled to the memory. The one or moreprocessors are configured to execute the instructions to determine,based on data descriptive of two or more audio environments, a geometryof a mutual audio environment. The one or more processors are alsoconfigured to process audio data, based on the geometry of the mutualaudio environment, for output at an audio device disposed in a firstaudio environment of the two or more audio environments.

In a particular aspect, a method includes determining, based on datadescriptive of two or more audio environments, a geometry of a mutualaudio environment. The method also includes processing audio data, basedon the geometry of the mutual audio environment, for output at an audiodevice disposed in a first audio environment of the two or more audioenvironments.

In a particular aspect, a non-transitory computer-readable storagemedium includes instructions that when executed by a processor, causethe processor to determine, based on data descriptive of two or moreaudio environments, a geometry of a mutual audio environment. Theinstructions, when executed by the processor, also cause the processorto process audio data, based on the geometry of the mutual audioenvironment, for output at an audio device disposed in a first audioenvironment of the two or more audio environments.

In a particular aspect, an apparatus for communication includes meansfor determining, based on data descriptive of two or more audioenvironments, a geometry of a mutual audio environment. The apparatusalso includes means for processing audio data, based on the geometry ofthe mutual audio environment, for output at an audio device disposed ina first audio environment of the two or more audio environments.

Other aspects, advantages, and features of the present disclosure willbecome apparent after review of the entire application, including thefollowing sections: Brief Description of the Drawings, DetailedDescription, and the Claims.

IV. BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example of a system that includes adevice that is configured to augment audio for communications, inaccordance with some examples of the present disclosure.

FIG. 2 is a block diagram of an example of a plurality of wirelessdevices that use RF sensing techniques to detect objects within aplurality of audio environments to generate a mapping of the audioenvironments, in accordance with some examples of the presentdisclosure.

FIG. 3 is a diagram illustrating an indoor environment that can includeone or more wireless devices configured to perform RF sensing to createan indoor map for use in augmenting communication, in accordance withsome examples of the present disclosure.

FIG. 4A illustrates a first exemplary audio environment configurationincluding a first audio environment, a second audio environment, and avirtual, mutual audio environment, in accordance with some examples ofthe present disclosure.

FIG. 4B illustrates a second exemplary audio environment configurationincluding a first audio environment, a second audio environment, and avirtual, mutual audio environment, in accordance with some examples ofthe present disclosure.

FIG. 4C illustrates a third exemplary audio environment configurationincluding a first audio environment, a second audio environment, and avirtual, mutual audio environment, in accordance with some examples ofthe present disclosure.

FIG. 4D illustrates a fourth exemplary audio environment configurationincluding a first audio environment, a second audio environment, and avirtual, mutual audio environment, in accordance with some examples ofthe present disclosure.

FIG. 5 is a flow chart of an example of a method for augmenting audiofor communications, in accordance with some examples of the presentdisclosure.

FIG. 6 is a flow chart of another example of a method for augmentingaudio for communications, in accordance with some examples of thepresent disclosure.

FIG. 7 is a block diagram illustrating a particular example of thedevice of FIG. 1 , in accordance with some examples of the presentdisclosure.

FIG. 8 illustrates a vehicle that incorporates aspects of the device ofFIG. 1 , in accordance with some examples of the present disclosure.

FIG. 9 illustrates a headset that incorporates aspects of the device ofFIG. 1 , in accordance with some examples of the present disclosure.

FIG. 10 illustrates a wearable electronic device that incorporatesaspects of the device of FIG. 1 , in accordance with some examples ofthe present disclosure.

FIG. 11 illustrates a voice-controlled speaker system that incorporatesaspects of the device of FIG. 1 , in accordance with some examples ofthe present disclosure.

FIG. 12 illustrates a camera that incorporates aspects of the device ofFIG. 1 , in accordance with some examples of the present disclosure.

FIG. 13 illustrates a mobile device that incorporates aspects of thedevice of FIG. 1 , in accordance with some examples of the presentdisclosure.

FIG. 14 illustrates a hearing aid device that incorporates aspects ofthe device of FIG. 1 , in accordance with some examples of the presentdisclosure.

FIG. 15 illustrates an aerial device that incorporates aspects of thedevice of FIG. 1 , in accordance with some examples of the presentdisclosure.

FIG. 16 illustrates a headset that incorporates aspects of the device ofFIG. 1 , in accordance with some examples of the present disclosure.

FIG. 17 illustrates an appliance that incorporates aspects of the deviceof FIG. 1 , in accordance with some examples of the present disclosure.

FIG. 18 is a flow chart of another example of a method for augmentingaudio for communications, in accordance with some examples of thepresent disclosure.

V. DETAILED DESCRIPTION

Systems to provide one or more virtual sound sources in an XRenvironment can generate sound in a manner that is unnatural for a userwhen compared to a user's experience of sound sources within the user'sreal-world environment. For example, current systems do not account fordiffering acoustical environments for different interacting users, nordo current systems account for the movement of a user within the XRenvironment with respect to the virtual sound sources.

The disclosed systems and methods determine, based on data descriptiveof two or more audio environments for two or more users, a geometry of amutual audio environment. The disclosed systems and methods can thenprocess audio data from the two or more users, based on the geometry ofthe mutual audio environment, for output at an audio device disposed ineach audio environment. For example, the disclosed systems and methodsuse various components of each user's networking environment (e.g., theuser's router, computing device, etc.) to determine the geometry of eachuser's audio environment and/or the position and orientation of eachuser within his/her audio environment. The disclosed systems and methodscan then generate a virtual, mutual audio environment for all users,based on the geometry of each user's audio environment, and then processeach user's audio data based on the geometry of the virtual, mutualaudio environment.

Particular aspects of the present disclosure are described below withreference to the drawings. In the description, common features aredesignated by common reference numbers. As used herein, variousterminology is used for the purpose of describing particularimplementations only and is not intended to be limiting ofimplementations. For example, the singular forms “a,” “an,” and “the”are intended to include the plural forms as well, unless the contextclearly indicates otherwise. Further, some features described herein aresingular in some implementations and plural in other implementations. Toillustrate, FIG. 1 depicts a device 100 that includes one or moreprocessors (e.g., processor(s) 102 in FIG. 1 ), which indicates that insome implementations the device 100 includes a single processor 102 andin other implementations the device 100 includes multiple processors102. For ease of reference herein, such features are generallyintroduced as “one or more” features and are subsequently referred to inthe singular or optional plural (generally indicated by terms ending in“(s)”) unless aspects related to multiple of the features are beingdescribed.

The terms “comprise,” “comprises,” and “comprising” are used hereininterchangeably with “include,” “includes,” or “including.”Additionally, the term “wherein” is used interchangeably with “where.”As used herein, “exemplary” indicates an example, an implementation,and/or an aspect, and should not be construed as limiting or asindicating a preference or a preferred implementation. As used herein,an ordinal term (e.g., “first,” “second,” “third,” etc.) used to modifyan element, such as a structure, a component, an operation, etc., doesnot by itself indicate any priority or order of the element with respectto another element, but rather merely distinguishes the element fromanother element having a same name (but for use of the ordinal term). Asused herein, the term “set” refers to one or more of a particularelement, and the term “plurality” refers to multiple (e.g., two or more)of a particular element.

As used herein, “coupled” may include “communicatively coupled,”“electrically coupled,” or “physically coupled,” and may also (oralternatively) include any combinations thereof. Two devices (orcomponents) may be coupled (e.g., communicatively coupled, electricallycoupled, or physically coupled) directly or indirectly via one or moreother devices, components, wires, buses, networks (e.g., a wirednetwork, a wireless network, or a combination thereof), etc. Two devices(or components) that are electrically coupled may be included in thesame device or in different devices and may be connected viaelectronics, one or more connectors, or inductive coupling, asillustrative, non-limiting examples. In some implementations, twodevices (or components) that are communicatively coupled, such as inelectrical communication, may send and receive electrical signals(digital signals or analog signals) directly or indirectly, such as viaone or more wires, buses, networks, etc. As used herein, “directlycoupled” refers to two devices that are coupled (e.g., communicativelycoupled, electrically coupled, or physically coupled) withoutintervening components.

In the present disclosure, terms such as “determining,” “calculating,”“estimating,” “shifting,” “adjusting,” etc. may be used to describe howone or more operations are performed. It should be noted that such termsare not to be construed as limiting and other techniques may be utilizedto perform similar operations. Additionally, as referred to herein,“generating,” “calculating,” “estimating,” “using,” “selecting,”“accessing,” and “determining” may be used interchangeably. For example,“generating,” “calculating,” “estimating,” or “determining” a parameter(or a signal) may refer to actively generating, estimating, calculating,or determining the parameter (or the signal) or may refer to using,selecting, or accessing the parameter (or signal) that is alreadygenerated, such as by another component or device.

FIG. 1 is a block diagram of an example of a system that includes adevice 100 that is configured to augment audio for communications, inaccordance with some examples of the present disclosure. The device 100can be used to augment audio from a first audio environment 124 foroutput in a second audio environment 126, where the first and secondaudio environments 124 and 126 have different physical geometries. Thedevice 100 can augment the audio from the first audio environment 124such that the audio sounds to the user of the second audio environment126 as though the first and second audio environments 124 and 126 havesimilar geometries.

In some implementations, the device 100 includes one or more processors102 coupled to a memory 104. The processor(s) 102 are configured toreceive a first audio environment description 114 descriptive of thefirst audio environment 124, to receive a second audio environmentdescription 116 descriptive of the second audio environment 126, and todetermine a mutual audio environment geometry 118 based on the first andsecond audio environment descriptions 114 and 116, as described furtherbelow. In some implementations, the processor(s) 102 are furtherconfigured to process, based on the mutual audio environment geometry118, first audio data 128 from the first audio environment 124 andsecond audio data 130 from the second audio environment 126 to generateprocessed first audio data 170 and processed second audio data 168, asdescribed further below.

In some implementations, the processor(s) 102 are further configured tooutput the processed first audio data 170 to the second audioenvironment 126 and/or output the processed second audio data 168 to thefirst audio environment 124. As described further below, the processedfirst audio data 170 can enable a user 134 in the second audioenvironment 126 to hear communication from the first audio environment124 as though the first and second audio environments 124 and 126 havesubstantially similar acoustical properties even when the real-worldacoustical properties (e.g., room dimensions, materials, etc.) of thefirst audio environment 124 and the second audio environment 126 aredifferent. Similarly, the processed second audio data 168 can enable auser 132 in the first audio environment 124 to hear communication fromthe second audio environment 126 as though the first and second audioenvironments 124 and 126 have substantially similar acousticalproperties.

In a particular aspect, the device 100 is disposed within the firstaudio environment 124 or the second audio environment 126. For example,the device 100 can include, correspond to, or be included within a userdevice 152 (such as a communication device or a computing device) usedby a user 132 in the first audio environment 124. In another particularaspect, the device 100 is remote from the first audio environment 124and the second audio environment 126. For example, the device 100 mayinclude, correspond to, or be included within one or more server devicesthat interact with user devices 152 and 154 disposed in the first andsecond audio environments 124 and 126.

The first audio environment 124 has particular acoustical propertiesbased on, for example, the physical dimensions of the first audioenvironment 124 (also referred to herein as the geometry of the firstaudio environment 124), the physical materials constituting the firstaudio environment 124, the acoustical materials of certain physicalmaterials constituting the first audio environment 124, and/or othercharacteristics of the first audio environment 124. For example, thefirst audio environment can be an eight-foot-by-eight-foot conferenceroom, with walls made of unpainted concrete with a sound absorptioncoefficient of 0.02.

In some implementations, the first audio environment 124 can include auser device 152. The user device 152 can include, correspond to, or beincluded within a communication device or a computing device such as adesktop computer, laptop computer, tablet, smart phone, etc. In someimplementations, the user device 152 can include one or more outputcomponents 160, such as a speaker, for outputting an audio signal to theuser 132 of the first audio environment 124. The user device 152 canalso be configured to convert incoming audio data to an audio signal foroutput to the user 132. For example, as described further below, theuser device 152 can convert the processed second audio data 168 from thedevice 100 into an audio signal for output to the user 132.

In the same or alternative implementations, the user device 152 can alsoinclude one or more input components 162, such as a microphone, forreceiving an audio signal from the user 132 of the first audioenvironment 124. The user device 152 can also be configured to convertan audio signal from the user 132 into data descriptive of the audiosignal for output to one or more other computing and/or communicationdevices. For example, as described further below, the user device 152can convert an incoming voice audio signal from the user 132 into thefirst audio data 128 for communication to the device 100.

The first audio environment 124 can also include one or more soundsources. The sound source(s) can include any source of sound that cangenerate an audio signal for communication to another audio environment.In a particular aspect, the sound source(s) can include or correspond tothe user 132 (e.g., by talking or otherwise making noise during aconference call), other people or animals in the first audio environment124, ambient sound sources (e.g., wind rustling leaves, traffic, etc.),and/or other devices in the first audio environment. To illustrate, twoor more users may be present in the first audio environment 124. Asanother illustrative example, the first audio environment 124 caninclude a plurality of instruments and/or vocalists performing in aconcert. The user device 152 can be configured to communicate datadescriptive of the sound source(s) to the device 100 for processing, asdescribed in more detail below and with reference to FIGS. 3-6 .

The audio signal from a sound source within the first audio environment124, such as the user 132, can travel myriad paths from the user 132 tothe user device 152, according to the geometry of the first audioenvironment 124. For example, as illustrated in FIG. 1 , the audiosignal from the user 132 can travel the first sound path 156 from theuser 132 to the user device 152. The exemplary first sound path 156reflects from the user 132 around a plurality of physical wallsconstituting the first audio environment 124 before arriving at the userdevice 152. Likewise, an output audio signal from the user device 152can travel the exemplary first sound path 156, reflecting around aplurality of physical walls constituting the first audio environment124, before arriving at the user 132. Depending on the path taken bysound in the first audio environment 124, the sound's acousticalproperties can change. For example, the audio signal can have a certainamount of reverberation. The reverberation associated with an audiosignal can include “early” reverberation and “late” reverberation. In aparticular example, early reverberation can include audio reflectionsheard by a listener within a threshold time of the sound's origination(e.g., 3-4 milliseconds). Late reverberation can include audioreflections heard by a listener outside the threshold time for earlyreverberation, but within a longer threshold time of the sound'sorigination (e.g., from 4-20 milliseconds).

Given the relatively constant speed of sound, the length of time ittakes an audio signal to traverse the first audio environment 124 canchange depending on the physical dimensions of the first audioenvironment 124. As a result, the amount and type of reverberationassociated with the audio signal can change. As the user 132 in thefirst audio environment 124 hears the reverberation associated with theaudio signal, the user's experience of the audio signal can change basedon the geometry of the first audio environment 124.

The second audio environment 126 has particular acoustical propertiesbased on, for example, the physical dimensions of the second audioenvironment 126 (also referred to herein as the geometry of the secondaudio environment 126), the physical materials constituting the secondaudio environment 126, the acoustical materials of certain physicalmaterials constituting the second audio environment 126, and/or otherdata characteristics of the second audio environment 126. For example,the second audio environment can be an eight-foot-by-twelve-footconference room, with walls made of painted concrete with a soundabsorption coefficient of 0.1.

The second audio environment 126 can also include a user device 154 insome implementations. The user device 154 can include, correspond to, orbe included within a communication device or a computing device such asa desktop computer, laptop computer, tablet, smart phone, etc. In someimplementations, the user device 154 can include one or more outputcomponents 166, such as a speaker, for outputting an audio signal to theuser 134 of the second audio environment 126. The user device 154 canalso be configured to convert incoming audio data to an audio signal foroutput to the user 134. For example, as described further below, theuser device 154 can convert the processed first audio data 170 from thedevice 100 into an audio signal for output to the user 134.

In the same or alternative implementations, the user device 154 can alsoinclude one or more input components 164, such as a microphone, forreceiving an audio signal from the user 134 of the second audioenvironment 126. The user device 154 can also be configured to convertan audio signal from the user 134 into data descriptive of the audiosignal for output to one or more other computing and/or communicationdevices. For example, as described further below, the user device 154can convert an incoming voice audio signal from the user 134 into thesecond audio data 130 for communication to the device 100.

The second audio environment 126 can also include one or more soundsources. The sound source(s) can include any source of sound that cangenerate an audio signal for communication to another audio environment.In a particular aspect, the sound source(s) can include or correspond tothe user 134 (e.g., by talking or otherwise making noise during aconference call), other people or animals in the second audioenvironment 126, ambient sound sources (e.g., wind rustling leaves,traffic, etc.), and/or other devices in the second audio environment. Toillustrate, two or more users may be present in the second audioenvironment 126. As another illustrative example, the second audioenvironment 126 can include a plurality of instruments and/or vocalistsperforming in a concert. The user device 154 can be configured tocommunicate data descriptive of the sound source(s) to the device 100for processing, as described in more detail below and with reference toFIGS. 3-6 .

The audio signal from a sound source within the second audio environment126, such as the user 134, can travel myriad paths from the user 134 tothe user device 154, according to the geometry of the second audioenvironment 126. For example, as illustrated in FIG. 1 , the audiosignal from the user 134 can travel the second sound path 158 from theuser 134 to the user device 154. The exemplary second sound path 158reflects from the user 134 around a plurality of physical wallsconstituting the second audio environment 126 before arriving at theuser device 154. Likewise, an output audio signal from the user device154 can travel the exemplary second sound path 158, reflecting around aplurality of physical walls constituting the second audio environment126, before arriving at the user 134.

Given the relatively constant speed of sound, the length of time ittakes an audio signal to traverse the second audio environment 126 canchange depending on the physical dimensions of the second audioenvironment 126. As a result, the amount and type of reverberationassociated with the audio signal can change. As the user 132 in thesecond audio environment 126 hears the reverberation associated with theaudio signal, the user's experience of the audio signal can change basedon the geometry of the second audio environment 126.

In some implementations, the user(s) 132 and 134 of the first and secondaudio environments 124 and 126 can also physically move around withinhis/her respective audio environment. For example, the user 132 can movearound within the physical dimensions of the audio environment (e.g.,pacing, moving to the front of the room, etc.). As an additionalexample, the user 132 can physically move part of his/her body withinthe audio environment (e.g., turning his/her head, facing a soundsource, turning away from a sound source, etc.).

Physical movement within the audio environment can be a natural part ofthe user 132 interacting with the audio environment. The physicalmovement, however, can change a user's position and/or orientation withrespect to one or more sound sources within the audio environment.Change to a user's position and/or orientation with respect to one ormore sound sources within the audio environment can change the way theuser experiences the sound source(s). For example, a user can expectthat a sound source should grow louder when the user moves closer to thesound source and quieter when the user moves further away from the soundsource. Keeping track of the user's movement within the audioenvironment can be used to provide more natural augmented audio inmulti-user communication, as described further below.

In some implementations, user device 152 can communicate a first audioenvironment description 114 to the device 100. The first audioenvironment description 114 can include data descriptive of one or moresound sources within the first audio environment 124, the geometry ofthe first audio environment 124, the user orientation of a user 132within the first audio environment 124, the user motion of the user 132within the first audio environment 124, an indication of a location of asound source within the first audio environment 124, and/or a locationwithin the first audio environment 124 of a virtual reality object, anaugmented reality object, a mixed reality object, and/or an extendedreality object. In the same or alternative implementations, the firstaudio environment description 114 can include data descriptive ofreverberation characteristics of the first audio environment 124,locations of particular features within the first audio environment 124(such as a location of a display screen), etc. In other examples, thefirst audio environment description 114 includes less information thatillustrated in FIG. 1 . For example, when the first audio environment124 includes a stationary workstation, data descriptive of user motionwithin the first audio environment 124 may be omitted.

In some implementations, user device 152 can communicate a second audioenvironment description 116 to the device 100. The second audioenvironment description 116 can include data descriptive of one or moresound sources within the second audio environment 126, the geometry ofthe second audio environment 126, the user orientation of a user 132within the second audio environment 126, the user motion of the user 132within the second audio environment 126, an indication of a location ofa sound source within the second audio environment 126, and/or alocation within the second audio environment 126 of a virtual realityobject, an augmented reality object, a mixed reality object, and/or anextended reality object. In the same or alternative implementations, thesecond audio environment description 116 can include data descriptive ofreverberation characteristics of the second audio environment 126,locations of particular features within the second audio environment 126(such as a location of a display screen), etc. In other examples, thesecond audio environment description 116 includes less information thatillustrated in FIG. 1 . For example, when the second audio environment126 includes a stationary workstation, data descriptive of user motionwithin the second audio environment 126 may be omitted.

In some implementations, first and second audio environment descriptions114 and 116 describe respective audio environments that can havedifferent acoustical properties. For example, the first and second audioenvironments 124 and 126 can have different physical dimensions,different acoustical properties, etc. As noted above, therefore, theusers 132 and 134 of the first and second audio environments 124 and 126can have different experiences and/or expectations associated with audiosignals within his/her respective audio environment.

In some implementations, the processor(s) 102 can be configured toselect a virtual, mutual audio environment for all users in a particularcommunication. For example, the mutual audio environment selector 110can use the first audio environment description 114 and the second audioenvironment description 116 to select a virtual, mutual audioenvironment with a mutual audio environment geometry 118 that does notcorrespond to the first audio environment 124 and/or the second audioenvironment 126. The processor(s) 102 can be further configured toprocess audio from the sound source(s) within the first and second audioenvironments 124 and 126 such that the users 132 and 134 hear the audioas though the audio originated within the virtual, mutual audioenvironment rather than from the differing, physical first and/or secondaudio environments 124, 126.

As an illustrative example, FIG. 1 illustrates a selection of a virtual,mutual audio environment. FIG. 1 illustrates a first exemplary XRenvironment 136 and a second exemplary XR environment 142. The first XRenvironment 136 and the second XR environment 142 are provided to aid inunderstanding and are not intended to limit the scope of the presentdisclosure. Other exemplary configurations of determining a mutual audioenvironment geometry 118 based on the first audio environmentdescription 114 and the second audio environment description 116 aredescribed in more detail below with reference to FIGS. 4A-4D. In FIG. 1, the first exemplary XR environment 136 generally corresponds to thefirst audio environment 124, based on the first audio environmentdescription 114. The second exemplary XR environment 142 generallycorresponds to the second audio environment 126, based on the secondaudio environment description 116.

The first exemplary XR environment 136 illustrates the geometry 138 ofthe first audio environment 124, while the second exemplary XRenvironment 142 illustrates the geometry 144 of the second audioenvironment 126. Sound paths 146 and 148 illustrate the different pathssound generated by a sound source can take within the first audioenvironment 124 and the second audio environment 126, respectively. Asnoted above, the differing sound paths can result in audio signals withdifferent acoustical properties such as reverberation. The reverberationassociated with an audio signal originating in the first audioenvironment 124 can, for example, sound unnatural to a user in thesecond audio environment 126, and vice versa.

Based on the first and second audio environment descriptions 114, 116,the processor(s) 102 can select a mutual audio environment geometry 118for use within both the first and second audio environments 124 and 126.FIG. 1 illustrates an exemplary mutual audio environment 140 within thefirst and second exemplary XR environments 136 and 142. As describedfurther below, the processor(s) 102 can alter the audio data from thefirst and second audio environments 124 and 126 to sound as though theaudio signals originated within the exemplary mutual audio environment140. To a user within either the first exemplary XR environment 136 orthe second exemplary XR environment 142, the audio signal can sound asthough it follows a third sound path 150, which can sound more naturalto the user.

The first XR environment 136 and the second XR environment 142 areprovided to aid in understanding and are not intended to limit the scopeof the present disclosure. Other exemplary configurations of determininga mutual audio environment geometry 118 based on the first audioenvironment description 114 and the second audio environment description116 are described in more detail below with reference to 4.

In some implementations, the processor(s) 102 can receive and store(e.g., as audio data 106) the first and/or second audio data 128 and 130from the first and/or second audio environments 124 and 126. Theprocessor(s) 102 can be further configured to process the audio data 106to match the mutual audio environment geometry 118 by altering theacoustical properties of the received audio data. For example, theprocessor(s) 102 can remove reverberation associated with the originalaudio signal. The processor(s) can then model reverberation associatedwith the geometry of the exemplary mutual audio environment 140 togenerate and store simulated reverberation 108.

As described in more detail below with reference to FIGS. 6 and 18 , inaddition to identifying reverberation associated with the audio data inits original audio environment, the processor(s) 102 can be configuredto simulate reverberation associated with the audio data in the virtual,mutual audio environment. This can include, for example, modeling earlyreflections via a fast fourier transform convolution of the audio datawith a simulated impulse response for the mutual audio environment.Modeling late reflections can include applying one or more comb and/orall-pass infinite impulse response filters to the audio data.

In some implementations, reverberation data can include directionalityinformation as well as frequency response. Directionality informationcan indicate the direction from which a reverberation reflection isoriginating with respect to various aspects of the mutual audioenvironment geometry 118. For example, directionality information canindicate whether a reverberation reflection is incident to a user fromone or more “walls” of a virtual, mutual audio environment. Frequencyresponse can indicate which, if any, frequencies of an audio signal arereflecting at a certain point in time. As described in more detail belowwith reference to FIGS. 6 and 18 , processing audio data can includesimulating reverberation within the mutual audio environment geometry118. Data associated with the simulated reverberation 108 can be storedat the memory 104.

In some implementations, processing the audio data 106 can includeadding, removing, and/or modifying directionality information associatedwith a sound source to the audio data 106. In a particular configurationthe directionality information can be stored separately from, or as partof, the simulated reverberation 108. In the same or alternativeimplementations, processing the audio data 106 can include changing afrequency range associated with one or more audio components of theaudio data. For example, the processor(s) 102 can be configured tochange one or more frequencies of the audio signal that reflect at acertain point in time. In a particular configuration, the frequencyrange modification(s) can be stored separately from, or as part of, thesimulated reverberation 108.

In some implementations, processing the audio data 106 can also, oralternatively, include applying one or more audio filters 120 to add,remove, modify, and/or enhance one or more audio components of the audiodata 106. As described in more detail below with reference to FIGS. 6and 18 , the audio processor(s) 112 can, for example, apply one or moreleast mean squares filters to remove reverberation from the datarepresentative of an original audio signal. The processor(s) 102 canalso, for example, apply one or more comb and/or all-pass infiniteimpulse response filters to simulate late reverberation within themutual audio environment, as described in more detail below withreference to FIGS. 6 and 18 . As an additional example, the processor(s)102 can apply one or more filters to reduce and/or suppress backgroundnoise associated with one or more audio environments, enhance the soundquality of one or more sound sources, change the frequency of one ormore sound sources, etc. As a further example, the processor(s) 102 canadd and/or enhance background noise to an audio signal to simulate adifferent audio environment.

In some implementations, the processor(s) 102 can then communicate theprocessed audio data to the first and/or second audio environments 124and 126, as appropriate. In some implementations, the device 100 cancommunicate the processed audio data such that the processed audio datais convolved with the simulated reverberation 108. In the same oralternative implementations, the device 100 can communicate theprocessed audio data and/or the simulated reverberation 108 separately.

In some implementations, the device 100 can be further configured tocommunicate the processed first and second audio data 168 and 170 backto the originating audio environment for output to a user. For example,the device 100 can be configured to communicate the processed firstaudio data 170 to the first audio environment 124 and the processedsecond audio data 168 to the second audio environment 124. The userdevice 152 can be further configured to output the processed first audiodata 170 to the user 132 of the first audio environment 124, and theuser device 154 can be further configured to output the processed secondaudio data 168 to the user 134 of the second audio environment 126. Insuch implementations, the users 132 and/or 134 may hear the processedaudio data through an output component 160 and/or 166 such as, forexample, headphones. The user could then hear their own voice, forexample, as well as audio output from another environment as though bothaudio sources originated in the same audio environment.

In some implementations, communications to one or more of the particularaudio environments, such as the first audio environment 124 and/or thesecond audio environment 126, can occur via one or more interfaces 122.The interface(s) 122 can be, for example, a wireless 802.11 interfaceand/or a wired Ethernet interface for communication with a particularaudio environment.

Although FIG. 1 illustrates certain implementations, otherimplementations are possible without departing from the scope of thepresent disclosure. For example, FIG. 1 illustrates the exemplary mutualaudio environment 140 as having a different geometry from the firstexemplary XR environment 136 and a different orientation from the secondexemplary XR environment 142. In other implementations, the mutual audioenvironment 140 can have the same geometry and/or orientation as one ormore of the individual audio environments. In the same or alternativeimplementations, the mutual audio environment 140 can have differentacoustical properties (e.g., sound dampening) from one or more of theindividual audio environments while having substantially the samedimensions as one or more of the individual audio environments. Otherexemplary mutual audio environments are described in more detail belowwith reference to FIGS. 4A-4D. In the same or alternativeimplementations, the device 100 can select the mutual audio environmentgeometry 118 for more than two individual audio environments.

According to a particular aspect, the device 100 can receive anautomated mapping of an individual audio environment using radiofrequency (“RF”) sensing techniques to detect object(s) within theindividual audio environment. FIG. 2 is a block diagram of an example ofa plurality of wireless devices 200 and 224 that use RF sensingtechniques to detect objects 202 and 226 within a plurality of audioenvironments 124, 126 to generate a wireless range measurement of theaudio environments 124, 126. In some implementations, the wirelessdevices 200 and 224 can be mobile phone(s), wireless access point(s),and/or other device(s) that include at least one RF interface.

In some implementations, the wireless devices 200, 224 can include oneor more components for transmitting an RF signal. Additionally, thewireless devices 200, 224 can include one or more digital-to-analogconverters (“DAC”) 204, 228 for receiving a digital signal or waveformand converting it to an analog waveform. The analog signals output fromthe DACs 204, 228 can be provided to one or more RF transmitters (“RFTX”) 206, 230. Each of the RF transmitters 206, 230 can be a Wi-Fitransmitter, a 5G/New Radio (“NR”) transmitter, a Bluetooth™transmitter, or any other transmitter capable of transmitting an RFsignal (Bluetooth is a registered trademark of Bluetooth SIG, Inc. ofKirkland, Wash., USA).

The RF transmitters 206, 230 can be coupled to one or more transmittingantennas 212, 236. In some implementations, each of the transmittingantennas 212, 236 can be an omnidirectional antenna capable oftransmitting an RF signal in all directions. For example, thetransmitting antenna 212 can be an omnidirectional Wi-Fi antenna thatcan radiate Wi-Fi signals (e.g., 2.4 GHz, 5 GHz, 6 GHz, etc.) in a360-degree radiation pattern. In another example, the transmittingantenna 236 can be a directional antenna that transmits an RF signal ina particular direction.

In some examples, the wireless devices 200, 224 can also include one ormore components for receiving an RF signal. For example, the wirelessdevice 200 can include one or more receiving antennas 214, and thewireless device 224 can include one or more receiving antennas 238. Insome examples, the receiving antenna 214 can be an omnidirectionalantenna capable of receiving RF signals from multiple directions. Inother examples, the receiving antenna 238 can be a directional antennathat is configured to receive signals from a particular direction. Infurther examples, both the transmitting antenna 212 and the receivingantenna 214 can include multiple antennas (e.g., elements) configured asan antenna array (e.g., linear antenna array, 2-dimensional antennaarray, 3-dimensional antenna array, or any combination thereof).

The wireless devices 200, 224 can also include one or more RF receivers(“RF RX”) 210, 234 coupled to the receiving antennas 214, 238,respectively. The RF receivers 210, 234 can include one or more hardwareand/or software components for receiving a waveform such as a Wi-Fisignal, a Bluetooth™ signal, a 5G/NR signal, or any other RF signal. Insome implementations, the RF receivers 210, 234 can be coupled toanalog-to-digital converters (“ADCs”) 208, 232, respectively. The ADCs208, 232 can be configured to convert the received analog waveform intoa digital waveform that can be provided to a processor.

In one example, the wireless devices 200, 224 can implement RF sensingtechniques by causing transmission waveforms 216, 240 to be transmittedfrom the transmitting antennas 212, 236. Although the transmissionwaveforms 216 are illustrated as single lines, in some implementations,one or more of the transmission waveforms 216, 240 can be transmitted inall directions by an omnidirectional transmitting antenna. For example,the transmission waveform 216 can be a Wi-Fi waveform that istransmitted by a Wi-Fi transmitter in the wireless device 200. As anadditional example, the transmission waveform 216 can be implemented tohave a sequence that has certain autocorrelation properties. Forinstance, the transmission waveform 216 can include single-carrierZadoff sequences and/or can include symbols similar to orthogonalfrequency-division multiplexing (OFDM) and/or Long Training Field (LTF)symbols.

In some techniques, the wireless devices 200, 224 can further implementRF sensing techniques by performing transmit and receive functionsconcurrently. For example, the wireless device 200 can enable its RFreceiver 210 to receive the waveform 218 at or near the same time as itenables the RF transmitter 206 to transmit the transmission waveform216. The waveform 218 is a reflected portion of the transmissionwaveform 216 that has reflected from the object 202.

In some examples, transmission of a sequence or pattern that is includedin the transmission waveform 216 can be repeated continuously such thatthe sequence is transmitted a certain number of times and/or for acertain duration of time. For example, if the wireless device 200enables the RF receiver 210 after enabling the RF transmitter 206,repeating a transmission pattern in the transmission waveform 216 can beused to avoid missing the reception of any reflected signals.

By implementing simultaneous transmit and receive functionality, thewireless devices 200, 224 can receive any signals that correspond to thetransmission waveforms 216, 240. For example, the wireless devices 200,224 can receive signals that are reflected from reflectors (e.g.,objects or walls) within a particular detection range of the wirelessdevices 200, 224, such as the waveforms 218, 242 reflected from objects202 and 226, respectively. The wireless devices 200, 224 can alsoreceive leakage signals (e.g., transmission leakage signals 220 and 244,respectively) that are coupled directly from the transmitting antennas212, 236 to the receiving antennas 214, 238 without reflecting from anyobjects. In some implementations, one or more of the waveforms 218, 242can include multiple sequences that correspond to multiple copies of asequence that are included in the transmission waveforms 216, 240. In aparticular implementation, the wireless devices 200, 224 can combine themultiple sequences that are received by the RF receivers 210, 234 toimprove the signal to noise ratio.

The wireless devices 200, 224 can further implement RF sensingtechniques by obtaining RF sensing data that is associated with each ofthe received signals corresponding to the transmission waveforms 216,240. In some examples, the RF sensing data can include channel stateinformation (“CSI”) based on data relating to the direct paths (e.g.,the leakage signals 220, 244) of the transmission waveforms 216, 240,together with data relating to the reflected paths (e.g., the waveforms218, 242) that correspond to the transmission waveforms 216, 240.

In some techniques, RF sensing data (e.g., CSI data) can includeinformation that can be used to determine how one or more of thetransmission waveforms 216, 240 propagates from one or more of the RFtransmitters 206, 230 to one or more of the RF receivers 210, 234. RFsensing data can include data that corresponds to the effects on thetransmitted RF signals due to multi-path propagation, scattering,fading, and power decay with distance, or any combination thereof. Insome examples, RF sensing data can include imaginary data and real data(e.g., I/Q components) corresponding to each tone in the frequencydomain over a particular bandwidth.

In some examples, RF sensing data can be used to calculate distances andangles of arrival that correspond to reflected waveforms, such as thewaveforms 218, 242. In further examples, RF sensing data can also beused to detect motion, determine location, detect changes in location ormotion patterns, obtain channel estimation, or any combination thereof.In some cases, the distance and angle of arrival of the reflectedsignals can be used to identify the size and position of reflectors inthe surrounding environment (e.g., objects 202, 226) to generate anindoor map. In some implementations, RF sensing data can also be used toidentify transient objects that can be omitted from an indoor map (e.g.,humans or pets walking through an indoor environment).

One or more of the wireless devices 200, 224 can also be configured tocalculate distances and angles of arrival corresponding to reflectedwaveforms (e.g., the distance and angle of arrival corresponding to oneor more of the waveforms 218, 242) by utilizing signal processing,machine learning algorithms, using any other suitable technique, or anycombination thereof. In other examples, one or more of the wirelessdevices 200, 224 can send the RF sensing data to another computingdevice, such as a server, that can perform the calculations to obtainthe distance and angle of arrival corresponding to one or more of thewaveforms 218, 242 and/or other reflected waveforms.

In a particular example, the distance traveled by one or more of thewaveforms 218, 242 can be calculated by measuring the difference in timefrom reception of the leakage signal 220, 244 to the reception of thereflected signals. For example, the wireless device 200 can determine abaseline distance of zero that is based on the difference from the timethe wireless device 200 transmits the transmission waveform 216 to thetime it receives the leakage signal 220 (e.g., propagation delay). Thewireless device 200 can then determine a distance associated with thewaveform 218 based on the difference from the time the wireless device200 transmits the transmission waveform 216 to the time it receives thewaveform 218, which can then be adjusted according to the propagationdelay associated with the leakage signal 220. In doing so, the wirelessdevice 200 can determine the distance traveled by the waveform 218,which can be used to determine the distance of a reflector (e.g., theobject 202) that caused the reflection.

In additional examples, the angle of arrival of the waveform 218 can becalculated by measuring the time difference of arrival of the waveform218 between individual elements of a receive antenna array, such as thereceiving antenna 214. In some examples, the time difference of arrivalcan be calculated by measuring the difference in received phase at eachelement in the receive antenna array.

In further examples, the distance and the angle of arrival of thewaveform 218 can be used to determine the distance between the wirelessdevice 200 and the object 202 as well as the position of the object 202relative to the wireless device 200. The distance and the angle ofarrival of the waveform 218 can also be used to determine the size andshape of the object 202 that causes the reflection. For example, thewireless device 200 can utilize the calculated distance and angle ofarrival corresponding to the waveform 218 to determine the point atwhich the transmission waveform 216 reflected from the object 202. Thewireless device 200 can aggregate the reflection points for variousreflected signals to determine the size and shape of the object 202.

In the same or alternative implementations, the wireless device 224 canbe configured to implement functionality similar to that described abovewith reference to the wireless device 200. For example, the wirelessdevice 224 can also determine the distance traveled by the waveform 242,which can be used to determine the distance of a reflector (e.g., theobject 226) that caused the reflection. The waveform 242 is a reflectedportion of the transmission waveform 240 that has reflected from theobject 226. The wireless device 224 can also be configured to calculatethe angle of arrival of the waveform 242 by measuring the timedifference of arrival of the waveform 242 between individual elements ofa receive antenna array, such as the receiving antenna 238. The wirelessdevice 224 can also be configured to use the distance and the angle ofarrival of the waveform 242 to determine the distance between thewireless device 224 and the object 226 as well as the position of theobject 226 relative to the wireless device 224.

As noted above, the wireless devices 200, 224 can include mobile devicessuch as smartphones, laptops, tablets, etc. In some implementations, oneor more of the wireless devices 200, 224 can be configured to obtaindevice location data and device orientation data together with the RFsensing data. In a particular implementation, device location data anddevice orientation data can be used to determine or adjust the distanceand angle of arrival of a reflected signal (e.g., waveforms 218, 242).For example, a user may be holding the wireless device 200 and walkingthrough a room during the RF sensing process. In this instance, thewireless device 200 can have a first location and a first orientationwhen it transmits the transmission waveform 216 and can have a secondlocation and a second orientation when it receives the waveform 218. Thewireless device 200 can account for the change in location and thechange in orientation when it processes the RF sensing data to calculatethe distance and angle of arrival. For example, the location data, theorientation data, and the RF sensing data can be correlated based on atime stamp associated with each element of data. In some techniques, thecombination of the location data, the orientation data, and the RFsensing data can be used to determine the size and location of theobject 202.

In some implementations, device position data can be gathered by one ormore of the wireless devices 200, 224 using techniques that includeround trip time (“RTT”) measurements, passive positioning, angle ofarrival, received signal strength indicator (“RSSI”), CSI data, usingany other suitable technique, or any combination thereof. In furtherexamples, device orientation data can be obtained from electronicsensors on the wireless devices 200, 224, such as a gyroscope, anaccelerometer, a compass, a magnetometer, any other suitable sensor, orany combination thereof. For instance, a gyroscope on the wirelessdevice 200 can be used to detect or measure changes in orientation ofthe wireless device 200 (e.g., relative orientation) and a compass canbe used to detect or measure absolute orientation of the wireless device200. In some implementations, the position and/or orientation of thewireless device 200 can be used by the device 100 of FIG. 1 as a proxyfor the position and/or orientation of the user 132 of the first audioenvironment 124. Similarly, the position and/or orientation of thewireless device 224 can be used by the device 100 of FIG. 1 as a proxyfor the position and/or orientation of the user 134 of the second audioenvironment 126.

Although FIG. 2 illustrates two wireless devices 200, 224 withsubstantially similar configurations, more than two devices can be usedwithout departing from the scope of the present disclosure. Further,each of the two or more devices can be configured to perform some or allof the indoor mapping functionality using different components and/orparticular implementations. For example, the wireless device 200 can usean omnidirectional transmitting antenna 212, while the wireless device224 can use a directional transmitting antenna 236, without departingfrom the scope of the present disclosure. As an additional example, thewireless device 224 can be configured to track movement data (e.g.,through an internal gyroscope), while the wireless device 200 can beconfigured to not track movement data.

In some implementations, the wireless device 200, 224 can be configuredto communicate mapping data associated with each of the first and secondaudio environments 124, 126 to the device 100. For example, the wirelessdevice 200 can be configured to communicate the distance between thewireless device 200 and the object 202 within the first audioenvironment 124. In some implementations, the mapping data can becommunicated to the device 100 as some or all of the first audioenvironment description 114. Likewise, the wireless device 224 can beconfigured to communicate the distance between the wireless device 224and the object 226 within the second audio environment 126. In someimplementations, the mapping data can be communicated to the device 100as some or all of the second audio environment description 116.

As described in more detail above with reference to FIG. 1 , the device100 can be configured to determine, based on the mapping data, ageometry of a mutual audio environment. The device 100 can, in someimplementations, also be configured to process audio data, based on thegeometry of the mutual audio environment, for output at one or moreaudio devices disposed in the first and second audio environments 124,126. In some implementations, the device 100 can be configured tocommunicate the processed second audio data 168 to the first audioenvironment 124 and the processed first audio data 170 to the secondaudio environment 126, as described in more detail above.

In addition to identifying the location of one or more objects 202, 226within the first and second audio environments 124, 126, the first andsecond audio environment descriptions 114, 116 can include a morethorough mapping of the respective audio environments. For example, inaddition to identifying the location of a signal-reflecting object, anaudio environment description can include a mapping of the wallsconstituting the audio environment. FIG. 3 is a diagram illustrating anindoor environment 300 that can include one or more wireless devicesconfigured to perform RF sensing to create an indoor map for use inaugmenting communication, in accordance with some examples of thepresent disclosure. In some examples, the indoor environment 300 caninclude one or more wireless devices 302 (e.g., a mobile device) and/orone or more stationary wireless devices (e.g., access point (“AP”) 304)that can be configured to perform RF sensing to create an indoor map ofindoor environment 300.

Generally, the indoor environment 300 corresponds to the first audioenvironment 124 of FIG. 1 . Although FIG. 3 illustrates a single indoorenvironment 300, more indoor environments can be included withoutdeparting from the scope of the present disclosure. For example, thedevice 100 can receive a first audio environment description 114 fromthe indoor environment 300 and a second audio environment description116 from another indoor environment. Further, although FIG. 3illustrates a single wireless device 302 and a single AP 304, more,fewer, and/or different components may be present within the indoorenvironment 300 without departing from the scope of the presentdisclosure. For example, the indoor environment 300 can include one ormore APs 304, one or more wireless devices 302, zero APs 304 (and one ormore wireless devices 302), and/or zero wireless devices 302 (and one ormore APs 304).

FIG. 3 illustrates an exemplary mapping of the indoor environment 300using a single AP 304 and a single wireless device 302. In a particularimplementation, the AP 304 can be a Wi-Fi access point having a staticor fixed location within the indoor environment 300. Although the indoorenvironment 300 is illustrated as having an access point (e.g., AP 304),any type of stationary wireless device (e.g., desktop computer, wirelessprinter, camera, smart television, smart appliance, etc.) can beconfigured to perform the techniques described herein. In one example,the AP 304 can include hardware and/or software components that can beconfigured to simultaneously transmit and receive RF signals, such asthe components described herein with respect to the wireless device 200of FIG. 2 . For example, the AP 304 can include one or more antennasthat can be configured to transmit an RF signal (e.g., the transmittingantenna 306) and one or more antennas that can be configured to receivean RF signal (e.g., the receiving antenna 308). As noted with respect tothe wireless device 200, the AP 304 can include omnidirectional antennasand/or antenna arrays that are configured to transmit and receivesignals from any direction.

In one aspect, the AP 304 can transmit an RF signal 310 that can reflectoff various reflectors (e.g., static or dynamic objects located within ascene; structural element(s) such as walls, ceilings, or other barriers;and/or other objects) located in the indoor environment 300. Forexample, the RF signal 310 can reflect from a wall 322 and cause areflected signal 312 to be received by the AP 304 via the receivingantenna 308. Upon transmitting the RF signal 310, the AP 304 can alsoreceive a leakage signal 314 corresponding to a direct path from thetransmitting antenna 306 to the receiving antenna 308.

In some implementations, the AP 304 can obtain RF sensing dataassociated with the reflected signal 312. For example, RF sensing datacan include CSI data corresponding to the reflected signal 312. In aparticular implementation, the AP 304 can use the RF sensing data tocalculate a distance D₁ and an angle of arrival θ₁ corresponding to thereflected signal 312. For example, the AP 304 can determine the distanceD₁ by calculating a time of flight for the reflected signal 312 based onthe difference or phase shift between the leakage signal 314 and thereflected signal 312. In the same or alternative implementations, the AP304 can determine the angle of arrival θ₁ by utilizing an antenna arrayto receive the reflected signals and measuring the difference inreceived phase at elements of the antenna array.

In some implementations, the AP 304 can utilize the distance D₁ and anangle of arrival θ₁ corresponding to one or more reflected signals(e.g., the reflected signal 312) to identify the wall 322. In someimplementations, the AP 304 can generate a map of the indoor environment300 that includes data representative of the wall 322 (e.g., as some orall of the first audio environment description 114). In the same oralternative implementations, the AP 304 can communicate to the device100 data for modifying a map of the indoor environment 300. Further, inthe same or alternative implementations, the AP 304 can gather RFsensing data and provide the RF sensing data to another computing device(e.g., a server) for processing the calculations of time of flight andangle of arrival for the reflected signals.

In some implementations, the indoor environment 300 can also include thewireless device(s) 302. Although illustrated as a smart phone, thewireless device 302 can include, correspond to, or be included withinany type of mobile device such as a tablet, laptop, smartwatch, etc. Ina particular implementation, the wireless device 302 can be configuredto perform RF sensing to create or modify an indoor map pertaining tothe indoor environment 300.

In a particular implementation, the wireless device 302 can cause awaveform 316A to be transmitted via one or more of its RF transmitters.FIG. 3 illustrates the mobile device transmitting the waveform 316A at afirst time (denoted “t1”) and a first location (denoted “(x1, y1)”). Ina particular implementation, the wireless device 302 can move while itis RF sensing such that the wireless device 302 is in a second location(denoted “(x2, y2)”) at a second, later time (denoted “t2”). In aparticular example, the waveform 316A can reflect from an object 320,and the wireless device 302 can receive the resultant reflected waveform318A at time t2. In another particular example, the wavelength of thewaveform 316A can be such that the waveform 316A penetrates and/ortraverses the object 320, resulting in the waveform 316B, which reflectsfrom a wall 324. The reflection 318B from the wall 324 can likewisetraverse the object 320 and result in a reflected waveform 318C beingreceived by the wireless device 302 at a third, later time t3.

In some implementations, the wireless device 302 can gather RF sensingdata corresponding to the reflected waveforms 318A and 318C. In furtheraspects, the wireless device 302 can also capture device location dataand device orientation data that corresponds to the time (e.g., t1) atwhich the waveform 316A was transmitted and/or to the times at which thereflected waveforms 318A (e.g., t2) and 318C (e.g., t3) were received.

In some implementations, the wireless device 302 can utilize the RFsensing data to calculate time of flight and angle of arrival for eachof the reflected waveform 318A and 318C. In further examples, thewireless device 302 can utilize the location data and orientation datato account for the device's movement during the RF sensing process. In aparticular implementation, the wireless device 302 can utilize the timeof flight and angle of arrival for one or more of the reflectedwaveforms 318A, 318C to estimate a position of the wireless device atthe time the reflected waveform(s) is received by the wireless device302 (e.g., x2, x3, etc.). The wireless device 302 can be configured toestimate movement of the wireless device 302 based on, for example, adifference between an estimate of the position of the wireless device ata first time (e.g., x2) and an estimate of the position of the wirelessdevice 302 at a second time (e.g., x3).

In the same or alternative implementations, the time of flight of thereflected waveforms 318A and 318C can be adjusted based on the device'smovement towards the object 320 and the wall 324, respectively. Inanother example, the angle of arrival of the reflected waveforms 318Aand 318C can be adjusted based on the movement and orientation of thewireless device 302 at the time it transmitted the waveform 316A versusthe time the wireless device 302 received the reflected waveforms 318Aand 318C.

In some implementations, the wireless device 302 can utilize the time offlight, distance, angle of arrival, location data, orientation data, orsome combination thereof to determine a size and/or position of theobject 320 and/or the wall 324.

In some implementations, the wireless device 302 can use the distance,angle of arrival, location, and orientation data to create a map of theindoor environment 300 that includes references to the object 320 andthe wall 324. In the same or alternative implementations, the wirelessdevice 302 can use the RF sensing data to modify a partial map that itreceives from a computing device, such as the device 100 of FIG. 1 . Inother aspects, the wireless device 302 can send the RF sensing data to aserver for processing and creation of a map of the indoor environment300.

As an illustrative example, the AP 304 and the wireless device 302 canbe configured to implement a bistatic configuration in which thetransmit and receive functions are performed by different devices. Forexample, the AP 304 (and/or another device within the indoor environment300 that is static or stationary) can transmit an omnidirectional RFsignal that can include the signals 328A and 328B. As illustrated, thesignal 328A can travel directly (e.g., no reflections) from the AP 304to the wireless device 302. The signal 328B can reflect off of a wall326 and cause a corresponding reflected signal 328C to be received bythe wireless device 302.

As another example, the wireless device 302 can utilize RF sensing dataassociated with the direct signal path (e.g., the signal 328A) and thereflected signal path (e.g., the signal 328C) to identify the size andshape of reflectors (e.g., the wall 326). For instance, the wirelessdevice 302 can obtain, retrieve, and/or estimate location dataassociated with the AP 304. In some implementations, the wireless device302 can use location data associated with the AP 304 and RF sensing data(e.g., CSI data) to determine time of flight, distance, and/or angle ofarrival associated with signals transmitted by the AP 304 (e.g., directpath signals such as the signal 328A and reflected path signals such asthe signal 328C). In some cases, the wireless device 302 and the AP 304can further send and/or receive communication that can include dataassociated with the RF signal 328A and/or the reflected signal 328C(e.g., transmission time, sequence/pattern, time of arrival, time offlight, angle of arrival, etc.).

In some implementations, the AP 304 and/or the wireless device 302 canbe configured to implement a monostatic configuration in which thetransmit and receive functions are performed by the same device. Forexample, the AP 304 and/or the wireless device (and/or another devicewithin the indoor environment 300 that is static or stationary) canperform RF sensing techniques irrespective of their association witheach other or with a Wi-Fi network. For example, the wireless device 302can utilize its Wi-Fi transmitter and Wi-Fi receiver to perform RFsensing as discussed herein when it is not associated with any accesspoint or Wi-Fi network. In further examples, the AP 304 can perform RFsensing techniques regardless of whether it has any wireless devicesassociated with it.

In some implementations, the wireless device 302 and the AP 304 canexchange data relating to their respective indoor maps for the indoorenvironment 300 to create a map that includes references to allreflectors (e.g., static objects, dynamic objects, structural elements)detected by both the wireless device 302 and the AP 304. In the same oralternative implementations, the RF sensing data from the wirelessdevice 302 and the AP 304 can be sent to one or more servers that canaggregate the data to generate or modify a map.

As an illustrative example, a server device can obtain (e.g.,crowdsource) RF sensing data from a plurality of wireless deviceslocated within an indoor environment (e.g., the indoor environment 300).The server device can use the RF sensing data from multiple devices toidentify and classify different reflectors. For example, the serverdevice may determine that a reflector is a transient object (e.g., a petor a human walking through the environment) by using the RF sensing datato track movement of the object or by determining that datacorresponding to the object was temporal and/or not confirmed by RFsensing data from other wireless devices. In some implementations, theserver device can omit and/or remove references to transient objectsfrom an indoor map. In another example, a computing device may use RFsensing data from a plurality of wireless devices to determine that areflector corresponds to a structural element such as a door, a window,a wall, a floor, a ceiling, a roof, a column, a staircase, or anycombination thereof. In the same or alternative implementations, acomputing device can include a reference in a map that indicates a typeof structural element. In some cases, a computing device may use RFsensing data from a plurality of wireless devices to determine that areflector corresponds to a static object such as a piece of furniture,an appliance, a fixture (e.g., blinds/shades, ceiling fans, plants,rugs, lamps, etc.). In some examples, a computing device can include areference in an indoor map that indicates a type of static object.

In some implementations, the wireless device 302 and/or the AP 304 canbe configured to communicate mapping data associated with the indoorenvironment 300 to the device 100. For example, the wireless device 302and/or the AP 304 can be configured to communicate the distance betweenthe wireless device 302 and the object 320 within the indoor environment300. As an additional example, the wireless device 302 and/or the AP 304can be configured to communicate the distance between the AP 304 (and/orthe wireless device 302) and one or more of the walls 322, 324, and 326.In some implementations, the mapping data can be communicated to thedevice 100 as some or all of the first audio environment description114.

As described in more detail above with reference to FIG. 1 , the device100 can be configured to determine, based on the mapping data, ageometry of a mutual audio environment. The device 100 can, in someimplementations, also be configured to process audio data, based on thegeometry of the mutual audio environment, for output at one or moreaudio devices disposed in the indoor environment 300. The device 100 canalso be configured to communicate the processed second audio data 168 tothe indoor environment 300 (e.g., the first audio environment 124), asdescribed in more detail above with reference to FIG. 1 .

FIGS. 2-3 illustrate various implementations in which various electronicdevices (e.g., the wireless device 200, 224 of FIG. 2 , the wirelessdevice 302 of FIG. 3 , and/or the AP 304 of FIG. 3 ) can map an audioenvironment for use by the device 100 in selecting a virtual, mutualaudio environment (e.g., the mutual audio environment geometry 118 ofFIG. 1 ). As described in more detail above with reference to FIG. 1 ,the mutual audio environment selector 110 can select a mutual audioenvironment geometry for one or more audio environments, and the device100 can process audio data, based on the geometry of the mutualenvironment, for output at one or more of the audio environments (e.g.,the first and second audio environments 124 and 126 of FIG. 1 ). FIGS.4A-4C illustrate a plurality of exemplary audio environmentconfigurations including a virtual, mutual audio environment.

FIG. 4A illustrates a first exemplary audio environment configuration400A including a first audio environment 402A, a second audioenvironment 404A, and a virtual, mutual audio environment 406A, inaccordance with some examples of the present disclosure. Generally, thefirst audio environment 402A corresponds to the first audio environment124 of FIG. 1 , and the second audio environment 404A corresponds to thesecond audio environment 126 of FIG. 1 .

FIG. 4A illustrates the second audio environment 404A as smaller in sizethan the first audio environment 402A. Accordingly, the exemplary soundpath 410A along which sound can travel to the user of the second audioenvironment 404A is shorter than the exemplary sound path 408A alongwhich sound can travel to the user of the first audio environment 402A.The users of the first and second audio environments 402A and 404A can,therefore, experience sound differently due to the differing acousticalproperties of sound traveling along the different sound paths 408A and410A.

As described in more detail above, the device 100 of FIG. 1 candetermine a virtual, mutual audio environment 406A for the first and/orsecond audio environments 402A and 404A. In the illustrative example ofFIG. 4A, the dimensions of the second audio environment 404A are suchthat the second audio environment 404A could spatially fit within thedimensions of the first audio environment 402A. As described in moredetail above with reference to FIGS. 1-3 , an electronic device withinthe first audio environment 402A (e.g., the user device 152 of FIG. 1 )can communicate a first audio environment description 414A to the device100 of FIG. 1 , while the second audio environment 404A can communicatea second audio environment description 416A to the device 100 of FIG. 1.

In some implementations, the device 100 can determine that the mutualaudio environment 406A can have the same dimensions of one or more ofthe individual audio environments. For example, FIG. 4A illustrates themutual audio environment 406A as having the same dimensions as thesecond audio environment 404A. In a particular configuration, therefore,the device 100 can be configured to only process audio for output at thefirst audio environment 402A. For example, the device 100 cancommunicate processed second audio data 418A to the first audioenvironment 402A. As described in more detail above with reference toFIG. 1 , the processed second audio data 418A can have acousticalcharacteristics that allow the user of the first audio environment 402Ato hear the processed second audio data 418A as though the user of thefirst audio environment 402A were disposed within the mutual audioenvironment 406A rather than within the first audio environment 402A.Accordingly, the user of the first audio environment 402A can experiencesound that travels an exemplary sound path 412A (rather than the soundpath 408A), which is substantially similar to the exemplary sound path410A. Thus, users of both the first and second audio environments 402Aand 404A can experience sound having substantially similar acousticalcharacteristics, improving the natural experience of communicationbetween the exemplary audio environments.

FIG. 4B illustrates a second exemplary audio environment configuration400B including a first audio environment 402B, a second audioenvironment 404B, and a virtual, mutual audio environment 406B.Generally, the first audio environment 402B corresponds to the firstaudio environment 124 of FIG. 1 , and the second audio environment 404Bcorresponds to the second audio environment 126 of FIG. 1 .

While FIG. 4A illustrates the second audio environment 404A as smallerin size than the first audio environment 402A, FIG. 4B illustrates thefirst and second audio environments 402B and 404B as having dissimilargeometries where the dimensions of one audio environment cannot bereadily translated to the other audio environment. Further, theexemplary sound path 408B along which sound can travel to the user ofthe first audio environment 402B and the exemplary sound path 410B alongwhich sound can travel to the user of the second audio environment 404Bcan differ. The users of the first and second audio environments 402Band 404B can, therefore, experience sound differently due to thediffering acoustical properties of sound traveling along the differentsound paths 408B and 410B.

As described in more detail above, the device 100 of FIG. 1 candetermine a virtual, mutual audio environment 406B for the first and/orsecond audio environments 402B and 404B. In the illustrative example ofFIG. 4B, an electronic device within the first audio environment 402B(e.g., the user device 152 of FIG. 1 ) can communicate a first audioenvironment description 414B to the device 100 of FIG. 1 , while thesecond audio environment 404B can communicate a second audio environmentdescription 416B to the device 100 of FIG. 1 . In some implementations,the device 100 can determine a mutual audio environment geometry havingarbitrary dimensions that do not match the dimensions of either thefirst or second audio environments 402B, 404B. For example, FIG. 4Billustrates the mutual audio environment 406B as having the dimensionsof a substantially rectangular area that can fit within both the firstand second audio environments 402B and 404B. A substantially rectangulararea may be chosen to, for example, lessen the amount of processingresources needed to process the audio data.

In a particular configuration, therefore, the device 100 can beconfigured to process audio for output at both the first and secondaudio environments 402B and 404B according to the geometry of the mutualaudio environment 406B. For example, the device 100 can communicateprocessed second audio data 418B to the first audio environment 402B andprocessed first audio data 420B to the second audio environment 404B. Asdescribed in more detail above with reference to FIG. 1 , the processedfirst and second audio data 418B and 420B can have acousticalcharacteristics that allow the users of the first and second audioenvironments 402B and 404B to hear audio as though the users weredisposed within the mutual audio environment 406B rather than the user'srespective actual audio environment. Accordingly, the users of the firstand second audio environments 402B and 404B can experience sound thattravels an exemplary sound path 412B. Thus, users of both the first andsecond audio environments 402B and 404B can experience sound havingsubstantially similar acoustical characteristics, improving the naturalexperience of communication between the exemplary audio environments.

FIG. 4C illustrates a third exemplary audio environment configuration400C including a first audio environment 402C, a second audioenvironment 404C, and a virtual, mutual audio environment 406C, inaccordance with some examples of the present disclosure. Generally, thefirst audio environment 402C corresponds to the first audio environment124 of FIG. 1 , and the second audio environment 404C corresponds to thesecond audio environment 126 of FIG. 1 .

While FIG. 4A illustrates the use of the actual dimensions of the secondaudio environment 404A as the dimensions of the mutual audio environment406A, FIG. 4C illustrates the use of a mutual audio environment 406Cwith arbitrary dimensions to account for the differing positions of theusers of the first and second audio environments 402C and 404C. In FIG.4C, the exemplary sound path 408C along which sound can travel to theuser of the first audio environment 402C and the exemplary sound path410C along which sound can travel to the user of the second audioenvironment 404C can differ. The users of the first and second audioenvironments 402C and 404C can, therefore, experience sound differentlydue to the differing acoustical properties of sound traveling along thedifferent sound paths 408C and 410C.

As described in more detail above, the device 100 of FIG. 1 candetermine a virtual, mutual audio environment 406C for the first and/orsecond audio environments 402C and 404C. In the illustrative example ofFIG. 4C, an electronic device within the first audio environment 402C(e.g., the user device 152 of FIG. 1 ) can communicate a first audioenvironment description 414C to the device 100 of FIG. 1 , while thesecond audio environment 404C can communicate a second audio environmentdescription 416C to the device 100 of FIG. 1 . In some implementations,the device 100 can determine a mutual audio environment geometry havingarbitrary dimensions that neither match nor fit within the real-worlddimensions of either the first or second audio environments 402C, 404C.For example, FIG. 4C illustrates the mutual audio environment 406C ashaving the dimensions of a substantially rectangular area that liesoutside the physical dimensions of both the first and second audioenvironments 402C and 404C. A substantially rectangular area may bechosen to, for example, lessen the amount of processing resources neededto process the audio data.

In a particular configuration, therefore, the device 100 can beconfigured to process audio for output at both the first and secondaudio environments 402C and 404C according to the geometry of the mutualaudio environment 406C. For example, the device 100 can communicateprocessed second audio data 418C to the first audio environment 402C andprocessed first audio data 420C to the second audio environment 404C. Asdescribed in more detail above with reference to FIG. 1 , the processedfirst and second audio data 418C and 420C can have acousticalcharacteristics that allow the users of the first and second audioenvironments 402C and 404C to hear audio as though the users weredisposed within the mutual audio environment 406C rather than the user'srespective actual audio environment. Accordingly, the users of the firstand second audio environments 402C and 404C can experience sound thattravels an exemplary sound path 412C. Thus, users of both the first andsecond audio environments 402C and 404C can experience sound havingsubstantially similar acoustical characteristics, improving the naturalexperience of communication between the exemplary audio environments.

FIG. 4D illustrates a fourth exemplary audio environment configuration400D including a first audio environment 402D, a second audioenvironment 404D, and a virtual, mutual audio environment 406D, inaccordance with some examples of the present disclosure. Generally, thefirst audio environment 402D corresponds to the first audio environment124 of FIG. 1 , and the second audio environment 404D corresponds to thesecond audio environment 126 of FIG. 1 .

While FIG. 4A illustrates the use of the actual dimensions of the secondaudio environment 404A as the dimensions of the mutual audio environment406A, FIG. 4D illustrates the use of a mutual audio environment 406Dwith arbitrary dimensions to account for differing orientations of theusers with respect to the first and second audio environments 402D and404D, respectively. In FIG. 4D, the exemplary sound path 408D alongwhich sound can travel to the user of the first audio environment 402Dand the exemplary sound path 410D along which sound can travel to theuser of the second audio environment 404D can differ. The users of thefirst and second audio environments 402D and 404D can, therefore,experience sound differently due to the differing acoustical propertiesof sound traveling along the different sound paths 408D and 410D.

As described in more detail above, the device 100 of FIG. 1 candetermine a virtual, mutual audio environment 406D for the first and/orsecond audio environments 402D and 404D. In the illustrative example ofFIG. 4D, an electronic device within the first audio environment 402D(e.g., the user device 152 of FIG. 1 ) can communicate a first audioenvironment description 414D to the device 100 of FIG. 1 , while thesecond audio environment 404D can communicate a second audio environmentdescription 416D to the device 100 of FIG. 1 . In some implementations,the device 100 can determine a mutual audio environment geometry havingarbitrary dimensions that neither match nor fit within the real-worlddimensions of either the first or second audio environments 402D, 404D.For example, FIG. 4D illustrates the mutual audio environment 406D ashaving the dimensions of a substantially rectangular area that liesoutside the physical dimensions of both the first and second audioenvironments 402D and 404D. A substantially rectangular area may bechosen to, for example, lessen the amount of processing resources neededto process the audio data.

In a particular configuration, therefore, the device 100 can beconfigured to process audio for output at both the first and secondaudio environments 402D and 404D according to the geometry of the mutualaudio environment 406D. For example, the device 100 can communicateprocessed second audio data 418D to the first audio environment 402D andprocessed first audio data 420D to the second audio environment 404D. Asdescribed in more detail above with reference to FIG. 1 , the processedfirst and second audio data 418D and 420D can have acousticalcharacteristics that allow the users of the first and second audioenvironments 402D and 404D to hear audio as though the users weredisposed within the mutual audio environment 406D rather than the user'srespective actual audio environment. Accordingly, the users of the firstand second audio environments 402D and 404D can experience sound thattravels an exemplary sound path 412D. Thus, users of both the first andsecond audio environments 402D and 404D can experience sound havingsubstantially similar acoustical characteristics, improving the naturalexperience of communication between the exemplary audio environments.

FIGS. 4A-4D illustrate various exemplary audio environmentconfigurations that each include a first audio environment, a secondaudio environment, and a virtual, mutual audio environment. AlthoughFIGS. 4A-4D illustrate certain exemplary configurations, otherconfigurations are possible without departing from the scope of thepresent disclosure. For example, the mutual audio environment cancorrespond to a volume representing an intersection of the two or moreaudio environments. As an additional example, the mutual audioenvironment can correspond to a virtual space distinct from each of thetwo or more audio environments.

Further, determining the geometry of the mutual audio environment caninclude determining a mutual coordinate system based on the datadescriptive of the two or more audio environments. In someimplementations, the processor(s) 102 of FIG. 1 can associate a firstposition in the mutual coordinate system with a first sound source ofthe first audio environment 124 and associate a second position in themutual coordinate system with a second sound source in the second audioenvironment 126. In a particular implementation, the processor(s) 102 ofFIG. 1 can be further configured to map a gaze direction of a user in aparticular audio environment to the mutual coordinate system. The gazedirection of the user (e.g., the user 132 of the first audio environment124) can be determined by the user device 152 or from another suitablesource. The processor(s) 102 can then generate, based on the gazedirection, a visual rendering of the mutual audio environment. Bygenerating a visual rendering of the mutual audio environment, thedevice 100 of FIG. 1 can further enhance the experience of the user 132by allowing the user 132 to “see” the mutual audio environment as wellas processing audio data to appear to the user 132 as though the soundoriginated within the mutual audio environment.

Using a mutual coordinate system based on the data descriptive of thefirst and second audio environments 124, 126, the processor(s) 102 candetermine a geometry of a mutual audio environment that can facilitateaugmenting communications in both the first and second audioenvironments 124, 126.

As described in more detail above with reference to FIGS. 1-3 , thedevice 100 of FIG. 1 can be configured to process audio data, based onthe geometry of the mutual audio environment, for output at an audiodevice disposed in one or more of the audio environments. By augmentingaudio for communication in this manner, users within the one or moreaudio environments can experience audio in a more natural manner.

FIG. 5 is a flow chart of an example of a method 500 for augmentingaudio for communications, in accordance with some examples of thepresent disclosure. The method 500 may be initiated, performed, orcontrolled by one or more processors executing instructions, such as bythe processor(s) 102 of FIG. 1 executing instructions from the memory104.

In some implementations, the method 500 includes, at 502, determining,based on data descriptive of two or more audio environments, a geometryof a mutual audio environment. For example, the processor(s) 102 of FIG.1 can select the mutual audio environment geometry 118 based on thefirst audio environment description 114 and the second audio environmentdescription 116, as described in more detail above with reference toFIGS. 1-4D.

In the example of FIG. 5 , the method 500 also includes, at 504,processing audio data, based on the geometry of the mutual audioenvironment, for output at an audio device disposed in a first audioenvironment of the two or more audio environments. For example, theprocessor(s) of FIG. 1 can process the second audio data 130 from thesecond audio environment 126 according to the mutual audio environmentgeometry 118 to become the processed second audio data 168 for output tothe user device 152 disposed in the first audio environment 124. In aparticular example, the processor(s) of FIG. 1 can process the secondaudio data 130 based on the exemplary mutual audio environment 140.

In the example of FIG. 5 , the method 500 also includes, at 506,obtaining motion data indicating movement of a user within a secondaudio environment of the two or more audio environments, wherein theaudio data is modified based on the motion data. For example, theprocessor(s) 102 of FIG. 1 can obtain motion data indicating movement ofthe user 132 within the first audio environment 124. The processor(s)102 can be further configured to modify the first audio data 128 toaccount for the movement of the user 132 within the first audioenvironment 124. In a particular example, as described in more detailabove with reference to FIGS. 1 and 3 , the processor(s) 102 can modifythe first audio data to account for the movement of the user 132carrying the wireless device 302 of FIG. 3 within the first audioenvironment 124.

In the example of FIG. 5 , the method 500 also includes, at 508,modifying the mutual audio environment based on motion data associatedwith at least one of the two or more audio environments. For example,the processor(s) 102 of FIG. 1 can modify the exemplary mutual audioenvironment 140 based on motion data associated with the user 132 of thefirst audio environment 124. In a particular example, the processor(s)102 of FIG. 1 can modify the exemplary mutual audio environment 140 toaccount for the user's change in position and/or orientation relative tothe audio environment, as described in more detail above with referenceto FIGS. 4C-4D.

Although the method 500 is illustrated as including a certain number ofoperations, more, fewer, and/or different operations can be included inthe method 500 without departing from the scope of the presentdisclosure. For example, the method 500 can exclude obtaining motiondata indicating movement of a user, as described in more detail abovewith reference to FIGS. 1-4D. As an additional example, the method 500can vary depending on the number of audio environments participating ina particular communication.

In some implementations, the method 500 can repeat one or more of502-508 periodically and/or continuously. For example, the method 500can determine a geometry of a mutual audio environment after an elapsedtime (e.g., five seconds) to account for changes in a user's position,orientation, gaze angle, and/or other changes in the user's experienceof the acoustical properties of sound signals within the user's audioenvironment. In some implementations, the processor(s) 102 of FIG. 1 canbe configured to modify the mutual audio environment based on motiondata within the user's audio environment. Modifying the mutual audioenvironment can include, for example, shifting boundaries of the mutualaudio environment relative to one or more sound sources of the mutualaudio environment, changing a shape of the geometry of the mutual audioenvironment, changing a size of the mutual audio environment, and/orsome combination thereof.

Further, although the examples provided above in illustrating method 500include the processor(s) 102 of FIG. 1 performing operations of themethod 500, some or all of the operations of the method 500 can beperformed by any suitable computing device. For example, as describedabove with reference to 504, the method 500 can include processing audiodata, based on the geometry of the mutual audio environment, for outputat an audio device disposed in a first audio environment of the two ormore audio environments. In some configurations, the device 100 canprocess the audio data prior to communicating the processed audio datato the audio device for output. In the same or alternativeconfigurations, the audio device can process the audio data for outputafter receiving audio data from the device 100. As an additionalexample, the determination of the mutual audio environment geometry(e.g., as described above with reference to 502) can be performed by oneor more servers in communication with the processor(s) 102 of FIG. 1 .

FIG. 6 is a flow chart of another example of a method 600 for augmentingaudio for communications. The method 600 may be initiated, performed, orcontrolled by one or more processors executing instructions, such as bythe processor(s) 102 of FIG. 1 executing instructions from the memory104.

In the example of FIG. 6 , the method 600 includes, at 602, analyzingaudio environment geometries. In some implementations, analyzing theaudio environment geometries can include determining a virtual, mutualaudio environment. For example, as described in more detail above withreference to FIGS. 1-5 , the processor(s) 102 of FIG. 1 can analyze thefirst and second audio environment descriptions 114 and 116 from thefirst and second audio environments 124 and 126, respectively, todetermine a geometry associated with each of the first and second audioenvironments 124 and 126. The processor(s) 102 can be further configuredto select a mutual audio environment geometry 118 that can be used toaugment audio for communication in one or more of the first and secondaudio environments 124 and 126.

In the example of FIG. 6 , the method 600 also includes, at 604,determining whether the mutual audio environment geometry is the same asor substantially similar (e.g., based on one or more thresholds) to thegeometry of any audio environment (e.g., the geometry of the first audioenvironment 124 and/or the geometry of the second audio environment126). In the example of FIG. 6 , if the mutual audio environment is thesame as or substantially similar to the geometry of an audioenvironment, the method 600 can proceed to, at 606, select the audioenvironment geometry that is the same or substantially similar to themutual audio environment geometry.

In certain configurations, the method 600 can optionally include furtherprocessing, as illustrated by the dashed lines of FIG. 6 . For example,in the example of FIG. 6 , if the mutual audio environment geometry isnot the same as or substantially similar to the mutual audio environmentgeometry, the method 600 can also proceed to, at 608, estimate one ormore acoustical properties associated with the mutual audio environment(“MAE”) as described above.

In the example of FIG. 6 , the method 600 can also optionally include,at 608, estimating one or more acoustical properties associated with themutual audio environment (“MAE”). In some implementations, theacoustical properties can include reverberation associated with aparticular sound source within the mutual audio environment. Forexample, the processor(s) 102 of FIG. 1 can analyze one or more soundssources generating audio data (e.g., first audio data 128 and/or secondaudio data 130) and determine what reverberation characteristics theaudio data would have within the mutual audio environment. In someconfigurations, the processor(s) 102 can be configured to estimate thereverberation by calculating a room impulse response associated with theaudio data within the mutual audio environment using an image sourcemodel.

In the example of FIG. 6 , the method 600 can also optionally include,at 610, estimating certain acoustical properties associated with theaudio environments that do not have a geometry that is the same as orsubstantially similar to the geometry of the mutual audio environment.For example, the processor(s) 102 of FIG. 1 can estimate a reverberationassociated with the first audio data 128 from the first audioenvironment 124 if the geometry of the first audio environment 124 isnot the same as or substantially similar to the geometry of the mutualaudio environment. The processor(s) 102 can be configured to estimatethe reverberation by calculating a room impulse response associated withthe audio data within the mutual audio environment using an image sourcemodel.

In the example of FIG. 6 , the method 600 can also optionally include,at 612, processing audio data from the audio environments that do nothave a geometry that is the same as or substantially similar to thegeometry of the mutual audio environment to modify one or moreacoustical properties of the audio data (e.g., to remove reverberationcharacteristics). For example, the processor(s) 102 of FIG. 1 can beconfigured to process the first audio data 128 to remove anyreverberation associated with the first audio data 128 within the firstaudio environment 124 if the geometry of the first audio environment 124is not the same as or substantially similar to the geometry of themutual audio environment. In some configurations, this can include usingthe estimated reverberation described above with reference to 610.

In the example of FIG. 6 , the method 600 can also optionally include,at 614, adding the reverberation associated with the mutual audioenvironment to audio data associated with an audio environment that doesnot have a geometry that is the same as or substantially similar to thegeometry of the mutual audio environment. For example, the processor(s)102 of FIG. 1 can be configured to convolve one or more audio-alteringsignals (e.g., the reverberation associated with the mutual audioenvironment) with the processed audio data from an audio environment(e.g., the processed first audio data 170 if the first audio environment124 does not have a geometry that is the same as or substantiallysimilar to the geometry of the mutual audio environment). As anadditional example, the processor(s) 102 of FIG. 1 can be configured toreduce reverberation associated with one or more of the audioenvironments.

In the example of FIG. 6 , the method 600 can also optionally include,at 616, outputting the processed audio data to one or more users. Forexample, the processor(s) 102 of FIG. 1 can be configured to communicatethe processed first audio data 170 to the first audio environment 124and/or the second audio environment 126. In some configurations, theprocessor(s) 102 can be configured to communicate the processed firstaudio data 170 to the one or more output components 160 and/or outputcomponent(s) 166 (e.g., one or more speakers).

Although the method 600 is illustrated as including a certain number ofoperations, more, fewer, and/or different operations can be included inthe method 600 without departing from the scope of the presentdisclosure. For example, the method 600 can include a determination ofwhether to further alter the processed audio data, as described in moredetail below with reference to FIG. 18 . As an additional example, themethod 600 can vary depending on the number of audio environmentsparticipating in a particular communication. As a further example, themethod 600 can communicate a subset of processed audio data to allusers.

Further, although the examples provided above in illustrating method 600describe the processor(s) 102 of FIG. 1 performing the operations of themethod 600, some or all of operations of the method 600 can be performedby any suitable computing device. For example, the method 600 caninclude determining whether to further modify the processed audio dataprior to communicating the processed audio data to one or more users. Insome configurations, the device 100 can send the simulated reverberation108 separately from the processed audio data and a computing devicewithin the individual audio environment can further process thecommunicated audio data (e.g., by convolving the processed audio datawith the simulated reverberation). As an additional example, thedetermination of the mutual audio environment geometry (e.g., asdescribed above with reference to 602-606) can be performed by one ormore servers in communication with the processor(s) 102 of FIG. 1 .

FIG. 7 is a block diagram illustrating a particular example of thedevice 100 of FIG. 1 , in accordance with some examples of the presentdisclosure. In various implementations, the device 100 may have more orfewer components than illustrated in FIG. 7 .

In a particular implementation, the device 100 includes a processor 704(e.g., a central processing unit (CPU)). The device 100 may include oneor more additional processor(s) 706 (e.g., one or more digital signalprocessors (DSPs)). The processor 704, the processor(s) 706, or both,may correspond to the one or more processors 102 of FIG. 1 . Forexample, in FIG. 7 , the processor(s) 706 include the mutual audioenvironment selector 110 and the audio processor(s) 112.

In FIG. 7 , the device 100 also includes the memory 104 and a CODEC 724.The memory 104 stores instructions 760 that are executable by theprocessor 704, and/or the processor(s) 706, to implement one or moreoperations described with reference to FIGS. 1-6 . In an example, thememory 104 corresponds to a non-transitory computer-readable medium thatstores the instructions 760 executable by the one or more processors102, and the instructions 760 include or correspond to (e.g., areexecutable by a processor to perform operations attributed to) themutual audio environment selector 110, the audio processor(s) 112, or acombination thereof. The memory 104 may also store the audio data 106(and/or the simulated reverberation 108 of FIG. 1 ).

In FIG. 7 , the input component(s) 162 and/or the output component(s)160 may be coupled to the CODEC 724. In the example illustrated in FIG.7 , the CODEC 724 includes a digital-to-analog converter (DAC 726) andan analog-to-digital converter (ADC 728). In a particularimplementation, the CODEC 724 receives analog signals from the inputcomponent(s) 162 (e.g., the first audio data 128 of FIG. 1 ), convertsthe analog signals to digital signals using the ADC 728, and providesthe digital signals to the processor(s) 706. In a particularimplementation, the processor(s) 706 provide digital signals to theCODEC 724, and the CODEC 724 converts the digital signals to analogsignals using the DAC 726 and provides the analog signals (e.g., theprocessed second audio data 168 of FIG. 1 ) to the output component(s)160.

In FIG. 7 , the device 100 also includes a display 720 coupled to adisplay controller 710. In some implementations, the device 100 alsoincludes a modem 712 coupled to a transceiver 714. In FIG. 7 , thetransceiver 714 is coupled to an antenna 716 to enable wirelesscommunication with other devices, such as the remote computing device718 (e.g., a server or network memory storing at least a portion of themutual audio environment geometry 118). For example, the modem 712 maybe configured to receive a portion of the audio data 106 (e.g., thesecond audio data 130) from the remote computing device 718 via wirelesstransmission. In other examples, the transceiver 714 is also, oralternatively, coupled to a communication port (e.g., an ethernet port)to enable wired communication with other devices, such as the remotecomputing device 718.

In a particular implementation, the device 100 is included in asystem-in-package or system-on-chip device 702. In a particularimplementation, the memory 104, the processor 704, the processor(s) 706,the display controller 710, the CODEC 724, the modem 712, and thetransceiver 714 are included in the system-in-package or system-on-chipdevice 702. In a particular implementation, a power supply 730 iscoupled to the system-in-package or system-on-chip device 702. Moreover,in a particular implementation, as illustrated in FIG. 7 , the display720, the input component(s) 162, the output component(s) 160, theantenna 716, and the power supply 730 are external to thesystem-in-package or system-on-chip device 702. In a particularimplementation, each of the display 720, the input component(s) 162, theoutput component(s) 160, the antenna 716, and the power supply 730 maybe coupled to a component of the system-in-package or system-on-chipdevice 702, such as an interface or a controller.

The device 100 may include, correspond to, or be included within a voiceactivated device, an audio device, a wireless speaker and voiceactivated device, a portable electronic device, a car, a vehicle, acomputing device, a communication device, an internet-of-things (IoT)device, a virtual reality (VR) device, an augmented reality (AR) device,a mixed reality (MR) device, a smart speaker, a mobile computing device,a mobile communication device, a smart phone, a cellular phone, a laptopcomputer, a computer, a tablet, a personal digital assistant, a displaydevice, a television, a gaming console, an appliance, a music player, aradio, a digital video player, a digital video disc (DVD) player, atuner, a camera, a navigation device, or any combination thereof. In aparticular aspect, the processor 704, the processor(s) 706, or acombination thereof, are included in an integrated circuit.

FIG. 8 illustrates an example of a vehicle 800 that incorporates aspectsof the device 100 of FIG. 1 . According to one implementation, thevehicle 800 is a self-driving car. According to other implementations,the vehicle 800 is a car, a truck, a motorcycle, an aircraft, a watervehicle, etc. In FIG. 8 , the vehicle 800 includes the display 720, thedevice 100, one or more input components 164, one or more outputcomponents 166, or some combination thereof. The one or more inputcomponents 164 and/or the one or more output components 166 areillustrated in dashed lines because they may or may not be visible tothe user of the vehicle 800. The device 100 can be integrated into thevehicle 800 or coupled to the vehicle 800. In some implementations, theinterior of the vehicle 800 generally corresponds to the first audioenvironment 124.

In a particular aspect, the device 100 is coupled to the display 720 andprovides an output to the display 720 responsive to the communicationbetween the user of the vehicle 800 and another user of another audioenvironment, such as indicating to the user the number and identity ofother users in the communication.

In a particular implementation, the input component(s) 164 can includeone or more microphones, as described in more detail above withreference to FIG. 1 . In the same or alternative particularimplementations, the output component(s) 166 can include one or morespeakers, as described in more detail above with reference to FIG. 1 .

Thus, the techniques described with respect to FIGS. 1-6 enable thedevice 100 of the vehicle 800 to augment audio for communication betweenor among a user of the vehicle 800 and other users in communication withthe user of the vehicle 800.

FIG. 9 illustrates a headset that incorporates aspects of the device ofFIG. 1 . FIG. 9 depicts an example of the device 100 coupled to orintegrated within a headset 902, such as a virtual reality headset, anaugmented reality headset, a mixed reality headset, an extended realityheadset, a head-mounted display, or a combination thereof. A visualinterface device is positioned in front of the user's eyes to enabledisplay of augmented reality, mixed reality, or virtual reality imagesor scenes to the user while the headset 902 is worn. In someimplementations, the headset 902 includes one or more input components164, one or more output components 166, or some combination thereof. Theone or more input components 164 and/or the one or more outputcomponents 166 are illustrated in dashed lines because they may or maynot be visible to the user of the headset 902. The device 100 can beintegrated into the headset 902 or coupled to the headset 902. In someimplementations, the area in which a user of the headset 902 operatesthe headset 902 generally corresponds to the first audio environment124.

In a particular implementation, the input component(s) 164 can includeone or more microphones, as described in more detail above withreference to FIG. 1 . In the same or alternative particularimplementations, the output component(s) 166 can include one or morespeakers, as described in more detail above with reference to FIG. 1 .Thus, the techniques described with respect to FIGS. 1-6 enable thedevice 100 of the headset 902 to augment audio for communication betweenor among a user of the headset 902 and other users in communication withthe user of the headset 902.

FIG. 10 illustrates a wearable electronic device 1002 that incorporatesaspects of the device 100 of FIG. 1 . FIG. 10 illustrates the wearableelectronic device 1002 as a “smart watch” that includes one or moreinput components 164, one or more output components 166, or somecombination thereof. The one or more input components 164 and/or the oneor more output components 166 are illustrated in dashed lines becausethey may or may not be visible to the user of the wearable electronicdevice 1002. The device 100 can be integrated into the wearableelectronic device 1002 or coupled to the wearable electronic device1002. In some implementations, the area in which a user of the wearableelectronic device 1002 operates the wearable electronic device 1002generally corresponds to the first audio environment 124.

In a particular implementation, the input component(s) 164 can includeone or more microphones, as described in more detail above withreference to FIG. 1 . In the same or alternative particularimplementations, the output component(s) 166 can include one or morespeakers, as described in more detail above with reference to FIG. 1 .Thus, the techniques described with respect to FIGS. 1-6 enable thedevice 100 of the wearable electronic device 1002 to augment audio forcommunication between or among a user of the wearable electronic device1002 and other users in communication with the user of the wearableelectronic device 1002.

FIG. 11 is an illustrative example of a voice-controlled speaker system1100 that incorporates aspects of the device of FIG. 1 . Thevoice-controlled speaker system 1100 can have wireless networkconnectivity and is configured to execute an assistant operation. InFIG. 11 , the device 100 is included in the voice-controlled speakersystem 1100. The voice-controlled speaker system 1100 also includes oneor more input components 164, one or more output components 166, or somecombination thereof. The one or more input components 164 and/or the oneor more output components 166 are illustrated in dashed lines becausethey may or may not be visible to the user of the voice-controlledspeaker system 1100. The device 100 can be integrated into thevoice-controlled speaker system 1100 or coupled to the voice-controlledspeaker system 1100. In some implementations, the area in which a userof the voice-controlled speaker system 1100 operates thevoice-controlled speaker system 1100 generally corresponds to the firstaudio environment 124.

In a particular implementation, the input component(s) 164 can includeone or more microphones, as described in more detail above withreference to FIG. 1 . In the same or alternative particularimplementations, the output component(s) 166 can include one or morespeakers, as described in more detail above with reference to FIG. 1 .Thus, the techniques described with respect to FIGS. 1-6 enable thedevice 100 of the voice-controlled speaker system 1100 to augment audiofor communication between or among a user of the voice-controlledspeaker system 1100 and other users in communication with the user ofthe voice-controlled speaker system 1100.

FIG. 12 illustrates a camera 1200 that incorporates aspects of thedevice 100 of FIG. 1 . In FIG. 12 , the device 100 is incorporated in orcoupled to the camera 1200. The camera 1200 includes one or more inputcomponents 164, one or more output components 166, or some combinationthereof. Additionally, the camera 1200 includes the device 100, which isconfigured to augment audio for communication between or among users. Ina particular implementation, the camera 1200 is a video cameraconfigured to augment audio in a virtual telecommunication session.

FIG. 13 illustrates a mobile device 1300 that incorporates aspects ofthe device 100 of FIG. 1 . In FIG. 13 , the device 100 is incorporatedin or coupled to the mobile device 1300. The mobile device 1300 includesone or more input components 164, one or more output components 166, orsome combination thereof. Additionally, the mobile device 1300 includesthe device 100, which is configured to augment audio for communicationbetween or among users. In a particular implementation, the mobiledevice 1300 includes software configured to operate a virtualtelecommunication session.

FIG. 14 illustrates a hearing aid device 1400 that incorporates aspectsof the device 100 of FIG. 1 . In FIG. 14 , the hearing aid device 1400includes or is coupled to the device 100 of FIG. 1 . The hearing aiddevice 1400 includes one or more input components 164, one or moreoutput components 166, or some combination thereof. During operation,the hearing aid device 1400 may process audio received from the inputcomponent(s) 164 for augmenting audio for the user of the hearing aiddevice 1400.

FIG. 15 illustrates an aerial device 1500 that incorporates aspects ofthe device 100 of FIG. 1 . In FIG. 15 , the aerial device 1500 includesor is coupled to the device 100 of FIG. 1 . The aerial device 1500 is amanned, unmanned, or remotely piloted aerial device (e.g., a packagedelivery drone). During operation, the aerial device 1500 may beconfigured to provide a mobile, centralized computing platform forprocessing audio received from a plurality of audio environments inorder to augment audio for communication between or among users of theplurality of audio environments.

FIG. 16 illustrates a headset 1600 that incorporates aspects of thedevice 100 of FIG. 1 . In FIG. 16 , the headset 1600 includes or iscoupled to the device 100 of FIG. 1 . The headset 1600 includes one ormore of the input component(s) 164 of FIG. 1 positioned to primarilycapture speech of a user. The headset 1600 may also include one or moreadditional microphone positioned to primarily capture environmentalsounds (e.g., for noise canceling operations) and one or more of theoutput components 166 of FIG. 1 . In a particular aspect, the headset1600 may process audio incoming from the user of the headset 1600 foraugmenting communication between or among the user of the headset 1600and other users in other audio environments.

FIG. 17 illustrates an appliance 1700 that incorporates aspects of thedevice 100 of FIG. 1 . In FIG. 17 , the appliance 1700 is a lamp;however, in other implementations, the appliance 1700 includes anotherInternet-of-Things appliance, such as a refrigerator, a coffee maker, anoven, another household appliance, etc. The appliance 1700 includes oris coupled to the device 100 of FIG. 1 . The appliance 1700 includes theone or more input components 164, one or more output components 166, orsome combination thereof. In a particular aspect, the appliance 1700 mayprocess audio received via the one or more input components foraugmenting audio in a communication between or among a user in thevicinity of the appliance 1700 and other users in the communication.

FIG. 18 is a flow chart of another example of a method 1800 foraugmenting audio for communications. The method 1800 may be initiated,performed, or controlled by one or more processors executinginstructions, such as by the processor(s) 102 of FIG. 1 executinginstructions from the memory 104.

In the example of FIG. 18 , the method 1800 includes, at 1806, analyzingaudio environment geometries. In some implementations, analyzing theaudio environment geometries can include determining a virtual, mutualaudio environment. For example, as described in more detail above withreference to FIGS. 1-6 , the processor(s) 102 of FIG. 1 can analyze thefirst and second audio environment descriptions 114 and 1118 from thefirst and second audio environments 124 and 126, respectively, todetermine a geometry associated with each of the first and second audioenvironments 124 and 126. The processor(s) 102 can be further configuredto select a mutual audio environment geometry 118 that can be used toaugment audio for communication in one or more of the first and secondaudio environments 124 and 126.

In the example of FIG. 18 , the method 1800 also includes, at 1808,determining whether the mutual audio environment geometry is the same asor substantially similar (e.g., based on one or more thresholds) to thegeometry of the first audio environment (“1AE”). For example, theprocessor(s) 102 of FIG. 1 can determine whether the mutual audioenvironment geometry 118 is the same as or substantially similar to thegeometry of the first audio environment 124. In the example of FIG. 18 ,if the mutual audio environment is the same as or substantially similarto the geometry of the first audio environment, the method 1800 canproceed to, at 1816, select the first audio environment geometry as themutual audio environment geometry.

In the example of FIG. 18 , if the first audio environment geometry isnot the same as or substantially similar to the mutual audio environmentgeometry, the method 1800 can also include, at 1810, determining whetherthe mutual audio environment geometry is the same as or substantiallysimilar to the geometry of the second audio environment (“2AE”). Forexample, the processor(s) 102 of FIG. 1 can determine whether the mutualaudio environment geometry 118 is the same as or substantially similarto (e.g., based on one or more thresholds) the geometry of the secondaudio environment 126. In the example of FIG. 18 , if the mutual audioenvironment is the same as or substantially similar to the geometry ofthe second audio environment, the method 1800 can proceed to, at 1818,select the second audio environment geometry as the mutual audioenvironment geometry.

In certain configurations, the method 1800 can optionally includefurther processing, as illustrated by the dashed lines of FIG. 18 . Forexample, in the example of FIG. 18 , the method 1800 also can optionallyinclude, at 1832, estimating one or more acoustical propertiesassociated with the mutual audio environment (“MAE”). In someimplementations, the acoustical properties can include reverberationassociated with a particular sound source within the mutual audioenvironment. For example, the processor(s) 102 of FIG. 1 can analyze oneor more sounds sources generating audio data (e.g., first audio data 128and/or second audio data 130) and determine what reverberationcharacteristics the audio data would have within the mutual audioenvironment. In some configurations, the processor(s) 102 can beconfigured to estimate the reverberation by calculating a room impulseresponse associated with the audio data within the mutual audioenvironment using an image source model.

In the example of FIG. 18 , the method 1800 can also optionally include,at 1824, estimating certain acoustical properties associated with one ormore sound sources within the first audio environment. For example, theprocessor(s) 102 of FIG. 1 can estimate a reverberation associated withthe first audio data 128 from the first audio environment 124. In someconfigurations, the processor(s) 102 can be configured to estimate thereverberation by calculating a room impulse response associated with theaudio data within the mutual audio environment using an image sourcemodel.

In the example of FIG. 18 , the method 1800 can also optionally include,at 1828, processing audio data from the first audio environment tomodify one or more acoustical properties of the audio data. For example,the processor(s) 102 of FIG. 1 can be configured to process the firstaudio data 128 to remove any reverberation associated with the firstaudio data 128 within the first audio environment 124. In someconfigurations, this can include using the estimated reverberationassociated with the first audio environment 124 described above withreference to 1824.

In the example of FIG. 18 , the method 1800 can also optionally include,at 1834, determining whether to further alter incoming audio data priorto communicating the processed audio data to another audio environment.For example, the processor(s) 102 of FIG. 1 can be configured todetermine whether to convolve one or more audio-altering signals withthe processed audio data. In some configurations, the processor(s) 102can be configured to determine whether to convolve the processed firstaudio data described above with reference to 1828 with a reverberationsignal associated with the mutual audio environment (described abovewith reference to 1832). As described in more detail above withreference to FIG. 1 , the device 100 can be configured to communicatethe processed first audio data 170, which can, in some implementations,include the simulated reverberation 108 associated with the mutual audioenvironment. In the same or alternative implementations, the processedfirst audio data 170 may not include the simulated reverberation 108,which can be communicated separately.

In the example of FIG. 18 , if the method 1800 determines to furtheralter incoming audio data, the method 1800 can also optionally include,at 1836, further altering the incoming audio data prior to communicatingthe processed audio data to another audio environment. For example, theprocessor(s) 102 of FIG. 1 can be configured to convolve one or moreaudio-altering signals (e.g., the reverberation associated with themutual audio environment) with the processed audio data. As anadditional example, the processor(s) 102 of FIG. 1 can be configured toreduce reverberation associated with one or more of the audioenvironments.

In the example of FIG. 18 , the method 1800 can also optionally include,at 1842, determining which users in which audio environments shouldreceive the processed audio data. For example, the processor(s) 102 ofFIG. 1 can determine whether the user 132 of the first audio environment124 and/or the user 134 of the second audio environment 126 shouldreceive the processed first audio data 170. In some implementations, theprocessor(s) 102 can be configured to determine which users in whichaudio environment should receive the processed audio data before orafter the processed audio data has been further altered to include newacoustical properties (as described above with reference to 1834 and1836).

In the example of FIG. 18 , the method 1800 can also optionally include,at 1826, estimating certain acoustical properties associated with one ormore sound sources within the second audio environment. For example, theprocessor(s) 102 of FIG. 1 can estimate a reverberation associated withthe second audio data 130 from the second audio environment 126. In someconfigurations, the processor(s) 102 can be configured to estimate thereverberation by calculating a room impulse response associated with theaudio data within the mutual audio environment using an image sourcemodel.

In the example of FIG. 18 , the method 1800 can also optionally include,at 1830, processing audio data from the first audio environment tomodify one or more acoustical properties of the audio data. For example,the processor(s) 102 of FIG. 1 can be configured to process the secondaudio data 130 to remove any reverberation associated with the secondaudio data 130 within the second audio environment 126. In someconfigurations, this can include using the estimated reverberationassociated with the second audio environment 126 described above withreference to 1826.

In the example of FIG. 18 , the method 1800 can also optionally include,at 1838, determining whether to further alter incoming audio data priorto communicating the processed audio data to another audio environment.For example, the processor(s) 102 of FIG. 1 can be configured todetermine whether to convolve one or more audio-altering signals withthe processed audio data. In some configurations, the processor(s) 102can be configured to determine whether to convolve the processed secondaudio data described above with reference to 1830 with a reverberationsignal associated with the mutual audio environment (described abovewith reference to 1832). As described in more detail above withreference to FIG. 1 , the device 100 can be configured to communicatethe processed second audio data 1188, which can, in someimplementations, include the simulated reverberation 108 associated withthe mutual audio environment. In the same or alternativeimplementations, the processed second audio data 1188 may not includethe simulated reverberation 108, which can be communicated separately.

In the example of FIG. 18 , if the method 1800 determines to furtheralter incoming audio data, the method 1800 can also optionally include,at 1840, further altering the incoming audio data prior to communicatingthe processed audio data to another audio environment. For example, theprocessor(s) 102 of FIG. 1 can be configured to convolve one or moreaudio-altering signals (e.g., the reverberation associated with themutual audio environment) with the processed audio data. As anadditional example, the processor(s) 102 of FIG. 1 can be configured toreduce reverberation associated with one or more of the audioenvironments.

In the example of FIG. 18 , the method 1800 can also optionally include,at 1842, determining which users in which audio environments shouldreceive the processed audio data. For example, the processor(s) 102 ofFIG. 1 can determine whether the user 132 of the first audio environment124 and/or the user 134 of the second audio environment 126 shouldreceive the processed first audio data 170. In some implementations, theprocessor(s) 102 can be configured to determine which users in whichaudio environment should receive the processed audio data before orafter the processed audio data has been further altered to include newacoustical properties (as described above with reference to 1838 and1840).

In the example of FIG. 18 , the method 1800 can also optionally include,at 1844, outputting the processed audio data to one or more users. Forexample, the processor(s) 102 of FIG. 1 can be configured to communicatethe processed first audio data 170 to the first audio environment 124and/or the second audio environment 126. In some configurations, theprocessor(s) 102 can be configured to communicate the processed firstaudio data 170 to the one or more output components 160 and/or outputcomponent(s) 166 (e.g., one or more speakers).

Although the method 1800 is illustrated as including a certain number ofoperations, more, fewer, and/or different operations can be included inthe method 1800 without departing from the scope of the presentdisclosure. For example, the method 1800 can exclude the determinationof whether to further alter the processed audio data, as described abovewith reference to 1834 and 1838. As an additional example, the method1800 can vary depending on the number of audio environmentsparticipating in a particular communication. As a further example, themethod 1800 can communicate all processed audio data to all users.

Further, although the examples provided above in illustrating method1800 describe the processor(s) 102 of FIG. 1 performing the operationsof the method 1800, some or all of operations of the method 1800 can beperformed by any suitable computing device. For example, as describedabove with reference to 1834 and 1838, the method 1800 can includedetermining whether to further modify the processed audio data prior tocommunicating the processed audio data to one or more users. In someconfigurations, the device 100 can send the simulated reverberation 108separately from the processed audio data and a computing device withinthe individual audio environment can further process the communicatedaudio data (e.g., by convolving the processed audio data with thesimulated reverberation). As an additional example, the determination ofthe mutual audio environment geometry (e.g., as described above withreference to 1806) can be performed by one or more servers incommunication with the processor(s) 102 of FIG. 1 . As a furtherexample, the determination of whether to output a particular processedaudio signal to a user within a particular audio environment (e.g., asdescribed above with reference to 1842) can be performed by an outputdevice within the individual audio environment.

In some implementations, the operations described with reference toFIGS. 5-6 and/or 18 are performed at the device 100 of FIG. 1 (e.g., atthe one or more processors 102). The device 100 may include, correspondto, or be included within a voice activated device, an audio device, awireless speaker and voice activated device, a portable electronicdevice, a car, a vehicle, a computing device, a communication device, aninternet-of-things (IoT) device, a virtual reality (VR) device, anaugmented reality (AR) device, a mixed reality (MR) device, a hearingaid device, a smart speaker, a mobile computing device, a mobilecommunication device, a smart phone, a cellular phone, a laptopcomputer, a computer, a tablet, a personal digital assistant, a displaydevice, a television, a gaming console, an appliance, a music player, aradio, a digital video player, a digital video disc (DVD) player, atuner, a camera, a navigation device, or any combination thereof. In aparticular aspect, the one or more processors 102, the memory 104, or acombination thereof, are included in an integrated circuit. Variousimplementations that include aspects of the device 100 are describedfurther with reference to FIGS. 7-17 .

Devices (e.g., those previously mentioned) may have both Bluetooth andWi-Fi capabilities, or other wireless means to communicate with eachother. Inter-networked devices may have wireless means to communicatewith each other and may also be connected based on different cellularcommunication systems, such as, a Long Term Evolution (LTE) system, aCode Division Multiple Access (CDMA) system, a Global System for MobileCommunications (GSM) system, a wireless local area network (WLAN)system, or some other wireless system. A CDMA system may implementWideband CDMA (WCDMA), CDMA 1×, Evolution-Data Optimized (EVDO), TimeDivision Synchronous CDMA (TD-SCDMA), or some other version of CDMA. Asused herein, “wireless” refers to one or more of the above-listedtechnologies, one or more other technologies that enable transfer ofinformation other than via wires, or a combination thereof.

As used herein, “downloading” and “uploading” a model includestransferring of data (e.g., compressed data) corresponding to the modelover a wired link, over a wireless link, or a combination thereof. Forexample, wireless local area networks (“WLANs”) may be used in place of,or in addition to, wired networks. Wireless technologies, such asBluetooth® (“Bluetooth”) and Wireless Fidelity “Wi-Fi” or variants ofWi-Fi (e.g. Wi-Fi Direct), enable high speed communications betweenmobile electronic devices (e.g., cellular phones, watches, headphones,remote controls, etc.) that are within relatively short distances of oneanother (e.g., 100 to 200 meters or less depending on the specificwireless technology). Wi-Fi is often used to connect and exchangeinformation between a device with an access point, (e.g. a router) anddevices that are Wi-Fi enabled. Examples of such devices are smarttelevisions, laptops, thermostats, personal assistant devices, homeautomation devices, wireless speakers and other similar devices.Similarly, Bluetooth is also used to couple devices together. Example ofsuch are mobile phones, computers, digital cameras, wireless headsets,keyboards, mice or other input peripherals, and similar devices.

In conjunction with the described implementations, an apparatus includesmeans for determining, based on data descriptive of two or more audioenvironments, a geometry of a mutual audio environment. For example, themeans for determining the geometry of the mutual audio environmentincludes the device 100, the processor(s) 102, the mutual audioenvironment selector 110, the wireless device 200, the wireless device224, the wireless device 302, the AP 304, one or more other circuits orcomponents configured to determine, based on data descriptive of two ormore audio environments, a geometry of a mutual audio environment, orany combination thereof.

The apparatus also includes means for processing audio data, based onthe geometry of the mutual audio environment, for output at an audiodevice disposed in a first audio environment of the two or more audioenvironments. For example, the means for processing the audio dataincludes the device 100, the processor(s) 102, the audio processor(s)112, the wireless device 200, the wireless device 224, the wirelessdevice 302, the AP 304, one or more other circuits or componentsconfigured to process audio data, based on the geometry of the mutualaudio environment, for output at an audio device disposed in a firstaudio environment of the two or more audio environments, or anycombination thereof.

Those of skill would further appreciate that the various illustrativelogical blocks, configurations, modules, circuits, and algorithm stepsdescribed in connection with the implementations disclosed herein may beimplemented as electronic hardware, computer software executed by aprocessor, or combinations of both. Various illustrative components,blocks, configurations, modules, circuits, and steps have been describedabove generally in terms of their functionality. Whether suchfunctionality is implemented as hardware or processor executableinstructions depends upon the particular application and designconstraints imposed on the overall system. Skilled artisans mayimplement the described functionality in varying ways for eachparticular application, such implementation decisions are not to beinterpreted as causing a departure from the scope of the presentdisclosure.

The steps of a method or algorithm described in connection with theimplementations disclosed herein may be embodied directly in hardware,in a software module executed by a processor, or in a combination of thetwo. A software module may reside in random access memory (RAM), flashmemory, read-only memory (ROM), programmable read-only memory (PROM),erasable programmable read-only memory (EPROM), electrically erasableprogrammable read-only memory (EEPROM), registers, hard disk, aremovable disk, a compact disc read-only memory (CD-ROM), or any otherform of non-transient storage medium known in the art. An exemplarystorage medium is coupled to the processor such that the processor mayread information from, and write information to, the storage medium. Inthe alternative, the storage medium may be integral to the processor.The processor and the storage medium may reside in anapplication-specific integrated circuit (ASIC). The ASIC may reside in acomputing device or a user terminal. In the alternative, the processorand the storage medium may reside as discrete components in a computingdevice or user terminal.

Particular aspects of the disclosure are described below in a first setof interrelated clauses:

Clause 1 includes a device including: a memory storing instructions; andone or more processors coupled to the memory and configured to executethe instructions to: determine, based on data descriptive of two or moreaudio environments, a geometry of a mutual audio environment; andprocess audio data, based on the geometry of the mutual audioenvironment, for output at an audio device disposed in a first audioenvironment of the two or more audio environments.

Clause 2 includes the device of Clause 1, wherein the one or moreprocessors are further configured to execute the instructions to obtainthe audio data from a second audio device that is disposed in a secondaudio environment of the two or more audio environments.

Clause 3 includes the device of any of Clauses 1-2, wherein the audiodata represents audio of a call between two or more users, and the firstaudio environment includes a room in which at least one of the two ormore users is located.

Clause 4 includes the device of any of Clauses 1-3, wherein the mutualaudio environment corresponds to one of the two or more audioenvironments.

Clause 5 includes the device of any of Clauses 1-4, wherein the mutualaudio environment corresponds to a volume representing an intersectionof the two or more audio environments.

Clause 6 includes the device of any of Clauses 1-4, wherein the mutualaudio environment corresponds to a virtual space distinct from each ofthe two or more audio environments.

Clause 7 includes the device of any of Clauses 1-4, wherein the mutualaudio environment corresponds to a virtual room having reverberationcharacteristics based on the geometry of the mutual audio environment.

Clause 8 includes the device of any of Clauses 1-7, wherein processingthe audio data includes filtering the audio data to reduce reverberationassociated with a second audio environment of the two or more audioenvironments.

Clause 9 includes the device of any of Clauses 1-8, wherein processingthe audio data includes combining the audio data with simulatedreverberation associated with the mutual audio environment.

Clause 10 includes the device of any of Clauses 1-9, wherein processingthe audio data includes suppressing background noise associated with asecond audio environment of the two or more audio environments.

Clause 11 includes the device of any of Clauses 1-10, wherein processingthe audio data includes adding, to the audio data, directionalityinformation associated with a sound source.

Clause 12 includes the device of any of Clauses 1-10, wherein processingthe audio data includes removing, from the audio data, directionalityinformation associated with a sound source.

Clause 13 includes the device of any of Clauses 1-10, wherein processingthe audio data includes modifying a portion of the audio data thatindicates directionality information associated with a sound source.

Clause 14 includes the device of any of Clauses 1-13, wherein the one ormore processors are further configured to execute the instructions toobtain motion data indicating movement of a user within a second audioenvironment of the two or more audio environments, wherein the audiodata is modified based on the motion data.

Clause 15 includes the device of Clause 14, wherein the motion data isbased on a change in a gaze direction of the user.

Clause 16 includes the device of any of Clauses 1-15, wherein the one ormore processors are further configured to execute the instructions tomodify the mutual audio environment based on motion data associated withat least one of the two or more audio environments.

Clause 17 includes the device of Clause 16, wherein modifying the mutualaudio environment includes shifting boundaries of the mutual audioenvironment relative to one or more sound sources of the mutual audioenvironment.

Clause 18 includes the device of Clause 16, wherein modifying the mutualaudio environment includes changing a shape of the geometry of themutual audio environment.

Clause 19 includes the device of Clause 16, wherein modifying the mutualaudio environment includes changing a size of the mutual audioenvironment.

Clause 20 includes the device of any of Clauses 1-19, wherein processingthe audio data includes removing one or more audio components from theaudio data.

Clause 21 includes the device of any of Clauses 1-19, wherein processingthe audio data includes adding one or more audio components to the audiodata.

Clause 22 includes the device of any of Clauses 1-19, wherein processingthe audio data includes modifying one or more audio components of theaudio data.

Clause 23 includes the device of any of Clauses 1-19, wherein processingthe audio data includes enhancing, in the audio data, one or more audiocomponents associated with a particular sound source.

Clause 24 includes the device of any of Clauses 1-23, wherein processingthe audio data includes changing a frequency range associated with oneor more audio components of the audio data.

Clause 25 includes the device of any of Clauses 1-24, wherein the one ormore processors are further configured to execute the instructions todetermine an orientation of a user relative to a sound source in themutual audio environment, wherein the audio data is processed based onthe orientation.

Clause 26 includes the device of Clause 25, wherein the one or moreprocessors are further configured to execute the instructions to obtainmotion data associated with at least one of the two or more audioenvironments and to modify the mutual audio environment, based on themotion data, to maintain the orientation.

Clause 27 includes the device of Clause 25, wherein the one or moreprocessors are further configured to execute the instructions to obtainmotion data associated with at least one of the two or more audioenvironments and to modify the orientation based on the motion data.

Clause 28 includes the device of any of Clauses 1-27, wherein the datadescriptive of the two or more audio environments includes geometry datafor each of the two or more audio environments.

Clause 29 includes the device of Clause 28, wherein the geometry dataassociated with a particular audio environment of the two or more audioenvironments is determined based on wireless range measurement of theparticular audio environment.

Clause 30 includes the device of any of Clauses 1-29, wherein the datadescriptive of the two or more audio environments includes reverberationcharacteristics data for each of the two or more audio environments.

Clause 31 includes the device of any of Clauses 1-30, wherein the datadescriptive of the two or more audio environments includes, for aparticular audio environment of the two or more audio environments, anindication of a location within the particular audio environment of asound source.

Clause 32 includes the device of any of Clauses 1-31, wherein the datadescriptive of the two or more audio environments includes, for aparticular audio environment of the two or more audio environments, anindication of a location within the particular audio environment of auser.

Clause 33 includes the device of any of Clauses 1-32, wherein audio datais processed further based on a location, within the mutual audioenvironment of a virtual reality object, an augmented reality object, amixed reality object, or an extended reality object.

Clause 34 includes the device of any of Clauses 1-33, whereindetermining the geometry of the mutual audio environment includesdetermining a mutual coordinate system based on the data descriptive ofthe two or more audio environments and the instructions are furtherconfigured to cause the one or more processors to associate a firstposition in the mutual coordinate system with a first sound source ofthe first audio environment and to associate a second position in themutual coordinate system with a second sound source of a second audioenvironment of the two or more audio environments.

Clause 35 includes the device of any of Clauses 1-34, whereindetermining the geometry of the mutual audio environment includesdetermining a mutual coordinate system based on the data descriptive ofthe two or more audio environments and the instructions are furtherconfigured to map a gaze direction of a user in a particular audioenvironment of the two or more audio environments to the mutualcoordinate system, and to generate, based on the gaze direction, avisual rendering of the mutual audio environment.

Clause 36 includes a method including: determining, based on datadescriptive of two or more audio environments, a geometry of a mutualaudio environment; and processing audio data, based on the geometry ofthe mutual audio environment, for output at an audio device disposed ina first audio environment of the two or more audio environments.

Clause 37 includes the method of Clause 36, further including obtainingthe audio data from a second audio device that is disposed in a secondaudio environment of the two or more audio environments.

Clause 38 includes the method of any of Clauses 36-37, wherein the audiodata represents audio of a call between two or more users, and the firstaudio environment includes a room in which at least one of the two ormore users is located.

Clause 39 includes the method of any of Clauses 36-38, wherein themutual audio environment corresponds to one of the two or more audioenvironments.

Clause 40 includes the method of any of Clauses 36-39, wherein themutual audio environment corresponds to a volume representing anintersection of the two or more audio environments.

Clause 41 includes the method of any of Clauses 36-39, wherein themutual audio environment corresponds to a virtual space distinct fromeach of the two or more audio environments.

Clause 42 includes the method of any of Clauses 36-39, wherein themutual audio environment corresponds to a virtual room havingreverberation characteristics based on the geometry of the mutual audioenvironment.

Clause 43 includes the method of any of Clauses 36-42, whereinprocessing the audio data includes filtering the audio data to reducereverberation associated with a second audio environment of the two ormore audio environments.

Clause 44 includes the method of any of Clauses 36-43, whereinprocessing the audio data includes combining the audio data withsimulated reverberation associated with the mutual audio environment.

Clause 45 includes the method of any of Clauses 36-44, whereinprocessing the audio data includes suppressing background noiseassociated with a second audio environment of the two or more audioenvironments.

Clause 46 includes the method of any of Clauses 36-45, whereinprocessing the audio data includes adding, to the audio data,directionality information associated with a sound source.

Clause 47 includes the method of any of Clauses 36-45, whereinprocessing the audio data includes removing, from the audio data,directionality information associated with a sound source.

Clause 48 includes the method of any of Clauses 36-45, whereinprocessing the audio data includes modifying a portion of the audio datathat indicates directionality information associated with a soundsource.

Clause 49 includes the method of any of Clauses 36-48, further includingobtaining motion data indicating movement of a user within a secondaudio environment of the two or more audio environments, wherein theaudio data is modified based on the motion data.

Clause 50 includes the method of Clause 49, wherein the motion data isbased on a change in a gaze direction of the user.

Clause 51 includes the method of any of Clauses 36-50, further includingmodifying the mutual audio environment based on motion data associatedwith at least one of the two or more audio environments.

Clause 52 includes the method of Clause 51, wherein modifying the mutualaudio environment includes shifting boundaries of the mutual audioenvironment relative to one or more sound sources of the mutual audioenvironment.

Clause 53 includes the method of Clause 51, wherein modifying the mutualaudio environment includes changing a shape of the geometry of themutual audio environment.

Clause 54 includes the method of Clause 51, wherein modifying the mutualaudio environment includes changing a size of the mutual audioenvironment.

Clause 55 includes the method of any of Clauses 36-54, whereinprocessing the audio data includes removing one or more audio componentsfrom the audio data.

Clause 56 includes the method of any of Clauses 36-54, whereinprocessing the audio data includes adding one or more audio componentsto the audio data.

Clause 57 includes the method of any of Clauses 36-54, whereinprocessing the audio data includes modifying one or more audiocomponents of the audio data.

Clause 58 includes the method of any of Clauses 36-54, whereinprocessing the audio data includes enhancing, in the audio data, one ormore audio components associated with a particular sound source.

Clause 59 includes the method of any of Clauses 36-58, whereinprocessing the audio data includes changing a frequency range associatedwith one or more audio components of the audio data.

Clause 60 includes the method of any of Clauses 36-59, further includingdetermining an orientation of a user relative to a sound source in themutual audio environment, wherein the audio data is processed based onthe orientation.

Clause 61 includes the method of Clause 61, further including obtainingmotion data associated with at least one of the two or more audioenvironments and modifying the mutual audio environment, based on themotion data, to maintain the orientation.

Clause 62 includes the method of Clause 61, further including obtainingmotion data associated with at least one of the two or more audioenvironments and modifying the orientation based on the motion data.

Clause 63 includes the method of any of Clauses 36-62, wherein the datadescriptive of the two or more audio environments includes geometry datafor each of the two or more audio environments.

Clause 64 includes the method of Clause 63, wherein the geometry dataassociated with a particular audio environment of the two or more audioenvironments is determined based on wireless range measurement of theparticular audio environment.

Clause 65 includes the method of any of Clauses 36-64, wherein the datadescriptive of the two or more audio environments includes reverberationcharacteristics data for each of the two or more audio environments.

Clause 66 includes the method of any of Clauses 36-65, wherein the datadescriptive of the two or more audio environments includes, for aparticular audio environment of the two or more audio environments, anindication of a location within the particular audio environment of asound source.

Clause 67 includes the method of any of Clauses 36-66, wherein the datadescriptive of the two or more audio environments includes, for aparticular audio environment of the two or more audio environments, anindication of a location within the particular audio environment of auser.

Clause 68 includes the method of any of Clauses 36-67, wherein audiodata is processed further based on a location, within the mutual audioenvironment of a virtual reality object, an augmented reality object, amixed reality object, or an extended reality object.

Clause 69 includes the method of any of Clauses 36-68, whereindetermining the geometry of the mutual audio environment includesdetermining a mutual coordinate system based on the data descriptive ofthe two or more audio environments and the method further includesassociating a first position in the mutual coordinate system with afirst sound source of the first audio environment and associating asecond position in the mutual coordinate system with a second soundsource of a second audio environment of the two or more audioenvironments.

Clause 70 includes the method of any of Clauses 36-69, whereindetermining the geometry of the mutual audio environment includesdetermining a mutual coordinate system based on the data descriptive ofthe two or more audio environments and the method further includesmapping a gaze direction of a user in a particular audio environment ofthe two or more audio environments to the mutual coordinate system, andgenerating, based on the gaze direction, a visual rendering of themutual audio environment.

Clause 71 includes a non-transient, computer-readable medium storinginstructions that, when executed by a processor, cause the processor to:determine, based on data descriptive of two or more audio environments,a geometry of a mutual audio environment; and process audio data, basedon the geometry of the mutual audio environment, for output at an audiodevice disposed in a first audio environment of the two or more audioenvironments.

Clause 72 includes the non-transient, computer-readable medium of Clause71, wherein the instructions, when executed by the processor, furthercause the processor to obtain the audio data from a second audio devicethat is disposed in a second audio environment of the two or more audioenvironments.

Clause 73 includes the non-transient, computer-readable medium of any ofClauses 71-72, wherein the audio data represents audio of a call betweentwo or more users, and the first audio environment includes a room inwhich at least one of the two or more users is located.

Clause 74 includes the non-transient, computer-readable medium of any ofClauses 71-73, wherein the mutual audio environment corresponds to oneof the two or more audio environments.

Clause 75 includes the non-transient, computer-readable medium of any ofClauses 71-74, wherein the mutual audio environment corresponds to avolume representing an intersection of the two or more audioenvironments.

Clause 76 includes the non-transient, computer-readable medium of any ofClauses 71-74, wherein the mutual audio environment corresponds to avirtual space distinct from each of the two or more audio environments.

Clause 77 includes the non-transient, computer-readable medium of any ofClauses 71-74, wherein the mutual audio environment corresponds to avirtual room having reverberation characteristics based on the geometryof the mutual audio environment.

Clause 78 includes the non-transient, computer-readable medium of any ofClauses 71-77, wherein processing the audio data includes filtering theaudio data to reduce reverberation associated with a second audioenvironment of the two or more audio environments.

Clause 79 includes the non-transient, computer-readable medium of any ofClauses 71-78, wherein processing the audio data includes combining theaudio data with simulated reverberation associated with the mutual audioenvironment.

Clause 80 includes the non-transient, computer-readable medium of any ofClauses 71-79, wherein processing the audio data includes suppressingbackground noise associated with a second audio environment of the twoor more audio environments.

Clause 81 includes the non-transient, computer-readable medium of any ofClauses 71-80, wherein processing the audio data includes adding, to theaudio data, directionality information associated with a sound source.

Clause 82 includes the non-transient, computer-readable medium of any ofClauses 71-80, wherein processing the audio data includes removing, fromthe audio data, directionality information associated with a soundsource.

Clause 83 includes the non-transient, computer-readable medium of any ofClauses 71-80, wherein processing the audio data includes modifying aportion of the audio data that indicates directionality informationassociated with a sound source.

Clause 84 includes the non-transient, computer-readable medium of any ofClauses 71-83, wherein the instructions, when executed by the processor,further cause the processor to obtain motion data indicating movement ofa user within a second audio environment of the two or more audioenvironments, wherein the audio data is modified based on the motiondata.

Clause 85 includes the non-transient, computer-readable medium of Clause84, wherein the motion data is based on a change in a gaze direction ofthe user.

Clause 86 includes the non-transient, computer-readable medium of any ofClauses 71-85, wherein the instructions, when executed by the processor,further cause the processor to modify the mutual audio environment basedon motion data associated with at least one of the two or more audioenvironments.

Clause 87 includes the non-transient, computer-readable medium of Clause86, wherein modifying the mutual audio environment includes shiftingboundaries of the mutual audio environment relative to one or more soundsources of the mutual audio environment.

Clause 88 includes the non-transient, computer-readable medium of Clause86, wherein modifying the mutual audio environment includes changing ashape of the geometry of the mutual audio environment.

Clause 89 includes the non-transient, computer-readable medium of Clause86, wherein modifying the mutual audio environment includes changing asize of the mutual audio environment.

Clause 90 includes the non-transient, computer-readable medium of any ofClauses 71-89, wherein processing the audio data includes removing oneor more audio components from the audio data.

Clause 91 includes the non-transient, computer-readable medium of any ofClauses 71-89, wherein processing the audio data includes adding one ormore audio components to the audio data.

Clause 92 includes the non-transient, computer-readable medium of any ofClauses 71-89, wherein processing the audio data includes modifying oneor more audio components of the audio data.

Clause 93 includes the non-transient, computer-readable medium of any ofClauses 71-89, wherein processing the audio data includes enhancing, inthe audio data, one or more audio components associated with aparticular sound source.

Clause 94 includes the non-transient, computer-readable medium of any ofClauses 71-93, wherein processing the audio data includes changing afrequency range associated with one or more audio components of theaudio data.

Clause 95 includes the non-transient, computer-readable medium of any ofClauses 71-94, wherein the instructions, when executed by the processor,further cause the processor to determine an orientation of a userrelative to a sound source in the mutual audio environment, wherein theaudio data is processed based on the orientation.

Clause 96 includes the non-transient, computer-readable medium of Clause95, wherein the instructions, when executed by the processor, furthercause the processor to obtain motion data associated with at least oneof the two or more audio environments and modifying the mutual audioenvironment, based on the motion data, to maintain the orientation.

Clause 97 includes the non-transient, computer-readable medium of Clause95, wherein the instructions, when executed by the processor, furthercause the processor to obtain motion data associated with at least oneof the two or more audio environments and modifying the orientationbased on the motion data.

Clause 98 includes the non-transient, computer-readable medium of any ofClauses 71-97, wherein the data descriptive of the two or more audioenvironments includes geometry data for each of the two or more audioenvironments.

Clause 99 includes the non-transient, computer-readable medium of Clause98, wherein the geometry data associated with a particular audioenvironment of the two or more audio environments is determined based onwireless range measurement of the particular audio environment.

Clause 100 includes the non-transient, computer-readable medium of anyof Clauses 71-99, wherein the data descriptive of the two or more audioenvironments includes reverberation characteristics data for each of thetwo or more audio environments.

Clause 101 includes the non-transient, computer-readable medium of anyof Clauses 71-100, wherein the data descriptive of the two or more audioenvironments includes, for a particular audio environment of the two ormore audio environments, an indication of a location within theparticular audio environment of a sound source.

Clause 102 includes the non-transient, computer-readable medium of anyof Clauses 71-101, wherein the data descriptive of the two or more audioenvironments includes, for a particular audio environment of the two ormore audio environments, an indication of a location within theparticular audio environment of a user.

Clause 103 includes the non-transient, computer-readable medium of anyof Clauses 71-102, wherein audio data is processed further based on alocation, within the mutual audio environment of a virtual realityobject, an augmented reality object, a mixed reality object, or anextended reality object.

Clause 104 includes the non-transient, computer-readable medium of anyof Clauses 71-103, wherein determining the geometry of the mutual audioenvironment includes determining a mutual coordinate system based on thedata descriptive of the two or more audio environments and thenon-transient, computer-readable medium further includes associating afirst position in the mutual coordinate system with a first sound sourceof the first audio environment and associating a second position in themutual coordinate system with a second sound source of a second audioenvironment of the two or more audio environments.

Clause 105 includes the non-transient, computer-readable medium of anyof Clauses 71-104, wherein determining the geometry of the mutual audioenvironment includes determining a mutual coordinate system based on thedata descriptive of the two or more audio environments and thenon-transient, computer-readable medium further includes mapping a gazedirection of a user in a particular audio environment of the two or moreaudio environments to the mutual coordinate system, and generating,based on the gaze direction, a visual rendering of the mutual audioenvironment.

Clause 105 includes a device including: means for determining, based ondata descriptive of two or more audio environments, a geometry of amutual audio environment; and means for processing audio data, based onthe geometry of the mutual audio environment, for output at an audiodevice disposed in a first audio environment of the two or more audioenvironments.

Clause 106 includes the device of Clause 105, further including meansfor obtaining the audio data from a second audio device that is disposedin a second audio environment of the two or more audio environments.

Clause 107 includes the device of any of Clauses 105-106, wherein theaudio data represents audio of a call between two or more users, and thefirst audio environment includes a room in which at least one of the twoor more users is located.

Clause 108 includes the device of any of Clauses 105-107, wherein themutual audio environment corresponds to one of the two or more audioenvironments.

Clause 109 includes the device of any of Clauses 105-108, wherein themutual audio environment corresponds to a volume representing anintersection of the two or more audio environments.

Clause 110 includes the device of any of Clauses 105-108, wherein themutual audio environment corresponds to a virtual space distinct fromeach of the two or more audio environments.

Clause 111 includes the device of any of Clauses 105-108, wherein themutual audio environment corresponds to a virtual room havingreverberation characteristics based on the geometry of the mutual audioenvironment.

Clause 112 includes the device of any of Clauses 105-111, whereinprocessing the audio data includes filtering the audio data to reducereverberation associated with a second audio environment of the two ormore audio environments.

Clause 113 includes the device of any of Clauses 105-112, whereinprocessing the audio data includes combining the audio data withsimulated reverberation associated with the mutual audio environment.

Clause 114 includes the device of any of Clauses 105-113, whereinprocessing the audio data includes suppressing background noiseassociated with a second audio environment of the two or more audioenvironments.

Clause 115 includes the device of any of Clauses 105-114, whereinprocessing the audio data includes adding, to the audio data,directionality information associated with a sound source.

Clause 116 includes the device of any of Clauses 105-114, whereinprocessing the audio data includes removing, from the audio data,directionality information associated with a sound source.

Clause 117 includes the device of any of Clauses 105-114, whereinprocessing the audio data includes modifying a portion of the audio datathat indicates directionality information associated with a soundsource.

Clause 118 includes the device of any of Clauses 105-117, furtherincluding means for obtaining motion data indicating movement of a userwithin a second audio environment of the two or more audio environments,wherein the audio data is modified based on the motion data.

Clause 119 includes the device of Clause 118, wherein the motion data isbased on a change in a gaze direction of the user.

Clause 120 includes the device of any of Clauses 105-119, furtherincluding means for modifying the mutual audio environment based onmotion data associated with at least one of the two or more audioenvironments.

Clause 121 includes the device of Clause 120, wherein modifying themutual audio environment includes shifting boundaries of the mutualaudio environment relative to one or more sound sources of the mutualaudio environment.

Clause 122 includes the device of Clause 120, wherein modifying themutual audio environment includes changing a shape of the geometry ofthe mutual audio environment.

Clause 123 includes the device of Clause 120, wherein modifying themutual audio environment includes changing a size of the mutual audioenvironment.

Clause 124 includes the device of any of Clauses 105-123, whereinprocessing the audio data includes removing one or more audio componentsfrom the audio data.

Clause 125 includes the device of any of Clauses 105-123, whereinprocessing the audio data includes adding one or more audio componentsto the audio data.

Clause 126 includes the device of any of Clauses 105-123, whereinprocessing the audio data includes modifying one or more audiocomponents of the audio data.

Clause 127 includes the device of any of Clauses 105-123, whereinprocessing the audio data includes enhancing, in the audio data, one ormore audio components associated with a particular sound source.

Clause 128 includes the device of any of Clauses 105-127, whereinprocessing the audio data includes changing a frequency range associatedwith one or more audio components of the audio data.

Clause 129 includes the device of any of Clauses 105-128, furtherincluding means for determining an orientation of a user relative to asound source in the mutual audio environment, wherein the audio data isprocessed based on the orientation.

Clause 130 includes the device of Clause 129, further including meansfor obtaining motion data associated with at least one of the two ormore audio environments and means for modifying the mutual audioenvironment, based on the motion data, to maintain the orientation.

Clause 131 includes the device of Clause 130, further including meansfor obtaining motion data associated with at least one of the two ormore audio environments and means for modifying the orientation based onthe motion data.

Clause 132 includes the device of any of Clauses 105-131, wherein thedata descriptive of the two or more audio environments includes geometrydata for each of the two or more audio environments.

Clause 133 includes the device of Clause 132, wherein the geometry dataassociated with a particular audio environment of the two or more audioenvironments is determined based on wireless range measurement of theparticular audio environment.

Clause 134 includes the device of any of Clauses 105-133, wherein thedata descriptive of the two or more audio environments includesreverberation characteristics data for each of the two or more audioenvironments.

Clause 135 includes the device of any of Clauses 105-134, wherein thedata descriptive of the two or more audio environments includes, for aparticular audio environment of the two or more audio environments, anindication of a location within the particular audio environment of asound source.

Clause 136 includes the device of any of Clauses 105-135, wherein thedata descriptive of the two or more audio environments includes, for aparticular audio environment of the two or more audio environments, anindication of a location within the particular audio environment of auser.

Clause 137 includes the device of any of Clauses 105-136, wherein audiodata is processed further based on a location, within the mutual audioenvironment of a virtual reality object, an augmented reality object, amixed reality object, or an extended reality object.

Clause 138 includes the device of any of Clauses 105-137, whereindetermining the geometry of the mutual audio environment includesdetermining a mutual coordinate system based on the data descriptive ofthe two or more audio environments and the device further includes meansfor associating a first position in the mutual coordinate system with afirst sound source of the first audio environment and associating asecond position in the mutual coordinate system with a second soundsource of a second audio environment of the two or more audioenvironments.

Clause 139 includes the device of any of Clauses 105-139, whereindetermining the geometry of the mutual audio environment includesdetermining a mutual coordinate system based on the data descriptive ofthe two or more audio environments and the device further includes meansfor mapping a gaze direction of a user in a particular audio environmentof the two or more audio environments to the mutual coordinate system,and generating, based on the gaze direction, a visual rendering of themutual audio environment.

Clause 140 includes the device of any of Clauses 1-35, further includinga modem coupled to the one or more processors and configured to send theprocessed audio data to the audio device.

What is claimed is:
 1. A device comprising: a memory configured to storedata descriptive of two or more audio environments; and one or moreprocessors coupled to the memory and configured to execute theinstructions to: determine, based on the data descriptive of two or moreaudio environments, a geometry of a mutual audio environment and amutual coordinate system; map a gaze direction of a user in a particularaudio environment of the two or more audio environments to the mutualcoordinate system; generate, based on the gaze direction, a visualrendering of the mutual audio environment; and process audio data, basedon the geometry of the mutual audio environment, for output at an audiodevice disposed in a first audio environment of the two or more audioenvironments.
 2. The device of claim 1, wherein the one or moreprocessors are further configured to obtain the audio data from a secondaudio device that is disposed in a second audio environment of the twoor more audio environments.
 3. The device of claim 1, wherein the audiodata represents audio of a call between two or more users, and the firstaudio environment includes a room in which at least one of the two ormore users is located.
 4. The device of claim 1, wherein the mutualaudio environment corresponds to one of the two or more audioenvironments, a volume representing an intersection of the two or moreaudio environments, a virtual space distinct from each of the two ormore audio environments, or a virtual room having reverberationcharacteristics based on the geometry of the mutual audio environment.5. The device of claim 1, wherein the one or more processors areconfigured to process the audio data comprises application of a filterto the audio data to reduce reverberation associated with a second audioenvironment of the two or more audio environments.
 6. The device ofclaim 1, wherein the one or more processors are configured to processthe audio data comprises: a combination of the audio data with simulatedreverberation associated with the mutual audio environment; suppressionof background noise associated with a second audio environment of thetwo or more audio environments; addition, to the audio data,directionality information associated with a sound source; removal, fromthe audio data, directionality information associated with the soundsource; and modification of a portion of the audio data that indicatesdirectionality information associated with the sound source.
 7. Thedevice of claim 1, wherein the one or more processors are furtherconfigured to obtain motion data indicating movement of a user within asecond audio environment of the two or more audio environments, whereinthe audio data is modified based on the motion data.
 8. The device ofclaim 7, wherein the motion data is based on a change in a gazedirection of the user.
 9. The device of claim 7, wherein the one or moreprocessors are configured to obtain motion data via a bistatic radiofrequency (“RF”) sensing operation.
 10. The device of claim 7, whereinthe one or more processors are configured to obtain motion data via amonostatic radio frequency (“RF”) sensing operation.
 11. The device ofclaim 1, wherein the one or more processors are further configured tomodify the mutual audio environment based on motion data associated withat least one of the two or more audio environments.
 12. The device ofclaim 11, wherein to modify the mutual audio environment includes one ormore shifts of boundaries of the mutual audio environment relative toone or more sound sources of the mutual audio environment, to change ashape of the geometry of the mutual audio environment, or to change asize of the mutual audio environment.
 13. The device of claim 1, whereinthe one or more processors are configured to process the audio datacomprises: removal of one or more audio components from the audio data;addition of one or more audio components to the audio data; andmodification of one or more audio components of the audio data.
 14. Thedevice of claim 1, wherein the one or more processors are configured toprocess the audio data comprises, in the audio data, enhancement of oneor more audio components associated with a particular sound source. 15.The device of claim 1, wherein the one or more processors are configuredto process the audio data comprises a change of frequency rangeassociated with one or more audio components of the audio data.
 16. Thedevice of claim 1, wherein the one or more processors are furtherconfigured to process the audio data based on determination of anorientation of a user relative to a sound source in the mutual audioenvironment.
 17. The device of claim 16, wherein the one or moreprocessors are further configured to obtain motion data associated withat least one of the two or more audio environments and to modify themutual audio environment, based on the motion data, to maintain theorientation.
 18. The device of claim 16, wherein the one or moreprocessors are further configured to obtain motion data associated withat least one of the two or more audio environments and to modify theorientation based on the motion data.
 19. The device of claim 1, whereinthe data descriptive of the two or more audio environments includesgeometry data for each of the two or more audio environments.
 20. Thedevice of claim 19, wherein the geometry data associated with aparticular audio environment of the two or more audio environments isdetermined based on wireless range measurement of the particular audioenvironment.
 21. The device of claim 1, wherein the data descriptive ofthe two or more audio environments includes reverberationcharacteristics data for each of the two or more audio environments. 22.The device of claim 1, wherein the data descriptive of the two or moreaudio environments includes, for a particular audio environment of thetwo or more audio environments, an indication of a location within theparticular audio environment of a sound source.
 23. The device of claim1, further comprising a modem coupled to the one or more processors andconfigured to send the processed audio data to the audio device.
 24. Thedevice of claim 1, wherein audio data is processed further based on alocation, within the mutual audio environment of a virtual realityobject, an augmented reality object, a mixed reality object, or anextended reality object.
 25. The device of claim 1, wherein the one ormore processors are configured to: determine the geometry of the mutualaudio environment comprises a mutual coordinate system based on the datadescriptive of the two or more audio environments; associate a firstposition in the mutual coordinate system with a first sound source ofthe first audio environment; and associate a second position in themutual coordinate system with a second sound source of a second audioenvironment of the two or more audio environments.
 26. A methodcomprising: determining, based on data descriptive of two or more audioenvironments, a geometry of a mutual audio environment and a mutualcoordinate system; mapping a gaze direction of a user in a particularaudio environment of the two or more audio environments to the mutualcoordinate system; generating, based on the gaze direction, a visualrendering of the mutual audio environment; and processing audio data,based on the geometry of the mutual audio environment, for output at anaudio device disposed in a first audio environment of the two or moreaudio environments.
 27. The method of claim 26, further comprisingobtaining motion data indicating movement of a user within a secondaudio environment of the two or more audio environments, wherein theaudio data is modified based on the motion data.
 28. A non-transient,computer-readable medium storing instructions that, when executed by aprocessor, cause the processor to: determine, based on data descriptiveof two or more audio environments, a geometry of a a mutual audioenvironment and a mutual coordinate system; map a gaze direction of auser in a particular audio environment of the two or more audioenvironments to the mutual coordinate system; generate, based on thegaze direction, a visual rendering of the mutual audio environment; andprocess audio data, based on the geometry of the mutual audioenvironment, for output at an audio device disposed in a first audioenvironment of the two or more audio environments.
 29. A devicecomprising: means for determining, based on data descriptive of two ormore audio environments, a geometry of a mutual audio environment and amutual coordinate system; means for mapping a gaze direction of a userin a particular audio environment of the two or more audio environmentsto the mutual coordinate system; means for generating, based on the gazedirection, a visual rendering of the mutual audio environment; and meansfor processing audio data, based on the geometry of the mutual audioenvironment, for output at an audio device disposed in a first audioenvironment of the two or more audio environments.